You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/05/13 16:58:24 UTC
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 58 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/58/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode
Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=5},status={state=notfound,msg=Did not find [search_rate_trigger3/5e1193b96c99cfT20t96hcplzrc0kzt4bkja07l1/0] in any tasks queue}}
Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=5},status={state=notfound,msg=Did not find [search_rate_trigger3/5e1193b96c99cfT20t96hcplzrc0kzt4bkja07l1/0] in any tasks queue}}
at __randomizedtesting.SeedInfo.seed([5E14A061AE531331:7C866EE399999C4C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:668)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<8> but was:<10>
Stack Trace:
java.lang.AssertionError: expected:<8> but was:<10>
at __randomizedtesting.SeedInfo.seed([5E14A061AE531331:33E8049C141BEC36]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:354)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13584 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/init-core-data-001
[junit4] 2> 1605452 WARN (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1605452 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1605454 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1605455 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1605455 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001
[junit4] 2> 1605455 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1605456 INFO (Thread-2134) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1605456 INFO (Thread-2134) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1605457 ERROR (Thread-2134) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1605556 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:59637
[junit4] 2> 1605562 INFO (zkConnectionManagerCallback-2851-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605569 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1605570 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1605577 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1605577 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1605577 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1605577 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24b35e12{/solr,null,AVAILABLE}
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7053552b{SSL,[ssl, http/1.1]}{127.0.0.1:54181}
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.e.j.s.Server Started @1605690ms
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54181}
[junit4] 2> 1605579 ERROR (jetty-launcher-2848-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1605579 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-13T15:41:00.226Z
[junit4] 2> 1605583 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1605583 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1605584 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1605584 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1605584 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1605584 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1605585 INFO (zkConnectionManagerCallback-2853-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605585 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1605588 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1605588 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1605588 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1605589 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e3696e3{/solr,null,AVAILABLE}
[junit4] 2> 1605589 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2092b777{SSL,[ssl, http/1.1]}{127.0.0.1:47805}
[junit4] 2> 1605589 INFO (jetty-launcher-2848-thread-2) [ ] o.e.j.s.Server Started @1605701ms
[junit4] 2> 1605589 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47805}
[junit4] 2> 1605590 ERROR (jetty-launcher-2848-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1605590 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1605590 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1605590 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1605590 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1605590 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-13T15:41:00.237Z
[junit4] 2> 1605593 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a90cfb8{/solr,null,AVAILABLE}
[junit4] 2> 1605593 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@85021e7{SSL,[ssl, http/1.1]}{127.0.0.1:57779}
[junit4] 2> 1605593 INFO (jetty-launcher-2848-thread-3) [ ] o.e.j.s.Server Started @1605704ms
[junit4] 2> 1605593 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57779}
[junit4] 2> 1605594 ERROR (jetty-launcher-2848-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1605594 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1605594 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1605594 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1605594 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1605594 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-13T15:41:00.241Z
[junit4] 2> 1605596 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1605596 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1605596 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1605596 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a8c9b92{/solr,null,AVAILABLE}
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@38760d0e{SSL,[ssl, http/1.1]}{127.0.0.1:39954}
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.e.j.s.Server Started @1605708ms
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39954}
[junit4] 2> 1605597 ERROR (jetty-launcher-2848-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1605597 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-13T15:41:00.244Z
[junit4] 2> 1605603 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1605603 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1605603 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1605603 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47bd7c1f{/solr,null,AVAILABLE}
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7f255eb2{SSL,[ssl, http/1.1]}{127.0.0.1:33014}
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.e.j.s.Server Started @1605715ms
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33014}
[junit4] 2> 1605604 ERROR (jetty-launcher-2848-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1605604 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-13T15:41:00.251Z
[junit4] 2> 1605607 INFO (zkConnectionManagerCallback-2857-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605607 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1605607 INFO (zkConnectionManagerCallback-2855-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605608 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1605628 INFO (zkConnectionManagerCallback-2859-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605628 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1605638 INFO (jetty-launcher-2848-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59637/solr
[junit4] 2> 1605639 INFO (zkConnectionManagerCallback-2861-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605639 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1605640 INFO (jetty-launcher-2848-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59637/solr
[junit4] 2> 1605650 INFO (jetty-launcher-2848-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59637/solr
[junit4] 2> 1605651 INFO (zkConnectionManagerCallback-2869-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605655 INFO (jetty-launcher-2848-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59637/solr
[junit4] 2> 1605657 INFO (jetty-launcher-2848-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59637/solr
[junit4] 2> 1605658 INFO (zkConnectionManagerCallback-2866-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605659 INFO (zkConnectionManagerCallback-2874-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605661 INFO (zkConnectionManagerCallback-2881-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605664 INFO (zkConnectionManagerCallback-2878-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605664 INFO (zkConnectionManagerCallback-2885-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605664 INFO (zkConnectionManagerCallback-2883-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605707 INFO (zkConnectionManagerCallback-2887-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605708 INFO (zkConnectionManagerCallback-2889-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605711 INFO (zkConnectionManagerCallback-2895-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605739 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1605739 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1605739 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1605745 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57779_solr
[junit4] 2> 1605747 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47805_solr
[junit4] 2> 1605747 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33014_solr
[junit4] 2> 1605748 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.Overseer Overseer (id=73792853437382669-127.0.0.1:57779_solr-n_0000000000) starting
[junit4] 2> 1605752 INFO (zkCallback-2894-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1605761 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1605774 INFO (zkCallback-2888-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1605795 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1605797 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1605799 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1605799 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54181_solr
[junit4] 2> 1605824 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1605826 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1605826 INFO (zkCallback-2894-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1605826 INFO (zkCallback-2888-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1605827 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1605827 INFO (zkCallback-2882-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1605828 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1605828 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39954_solr
[junit4] 2> 1605829 INFO (zkCallback-2888-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1605829 INFO (zkCallback-2894-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1605829 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1605844 INFO (zkConnectionManagerCallback-2906-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1605844 INFO (zkCallback-2882-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1605852 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1605853 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1605872 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1605887 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1605887 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57779_solr
[junit4] 2> 1605888 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605888 INFO (zkCallback-2882-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605888 INFO (zkCallback-2894-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605894 INFO (zkCallback-2888-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605894 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605894 INFO (zkCallback-2905-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1605908 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1605916 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1605917 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1605917 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1605924 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:47805_solr, 127.0.0.1:39954_solr, 127.0.0.1:54181_solr, 127.0.0.1:33014_solr, 127.0.0.1:57779_solr]
[junit4] 2> 1605936 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1605936 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1605937 DEBUG (ScheduledTrigger-3793-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1606100 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33014.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606146 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33014.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606146 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33014.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606147 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node5/.
[junit4] 2> 1606204 INFO (zkConnectionManagerCallback-2911-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606220 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606221 INFO (jetty-launcher-2848-thread-5) [n:127.0.0.1:33014_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1606283 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54181.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606297 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54181.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606297 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54181.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606317 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39954.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606324 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node1/.
[junit4] 2> 1606350 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39954.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606350 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39954.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606352 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node3/.
[junit4] 2> 1606370 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57779.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606383 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57779.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606383 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57779.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606384 INFO (zkConnectionManagerCallback-2916-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606385 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606385 INFO (jetty-launcher-2848-thread-3) [n:127.0.0.1:57779_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node4/.
[junit4] 2> 1606386 INFO (jetty-launcher-2848-thread-1) [n:127.0.0.1:54181_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1606389 INFO (zkConnectionManagerCallback-2921-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606390 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606391 INFO (jetty-launcher-2848-thread-4) [n:127.0.0.1:39954_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1606683 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47805.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606696 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47805.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606696 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47805.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1606697 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node2/.
[junit4] 2> 1606701 INFO (zkConnectionManagerCallback-2926-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606702 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606703 INFO (jetty-launcher-2848-thread-2) [n:127.0.0.1:47805_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1606768 INFO (zkConnectionManagerCallback-2928-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606771 INFO (zkConnectionManagerCallback-2933-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1606772 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606773 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[5E14A061AE531331]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59637/solr ready
[junit4] 2> 1606839 INFO (qtp1758755827-10146) [n:127.0.0.1:47805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1606875 INFO (qtp2002743592-10170) [n:127.0.0.1:33014_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1606883 INFO (qtp1832386451-10161) [n:127.0.0.1:39954_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1606902 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1606933 INFO (qtp571615543-10138) [n:127.0.0.1:57779_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1606933 DEBUG (qtp1832386451-10160) [n:127.0.0.1:39954_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1606934 DEBUG (zkCallback-2905-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1606935 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1606935 INFO (qtp1832386451-10160) [n:127.0.0.1:39954_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=126
[junit4] 2> 1606939 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1606939 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1606940 DEBUG (ScheduledTrigger-3793-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1606991 INFO (TEST-SearchRateTriggerIntegrationTest.testDeleteNode-seed#[5E14A061AE531331]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteNode
[junit4] 2> 1606992 INFO (TEST-SearchRateTriggerIntegrationTest.testDeleteNode-seed#[5E14A061AE531331]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 1606993 INFO (TEST-SearchRateTriggerIntegrationTest.testDeleteNode-seed#[5E14A061AE531331]) [ ] o.a.s.c.a.SearchRateTriggerIntegrationTest /autoscaling.json reset, new znode version 3
[junit4] 2> 1606993 DEBUG (zkCallback-2905-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1606993 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1606995 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1606995 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1606995 DEBUG (OverseerAutoScalingTriggerThread-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1612004 INFO (qtp1758755827-10121) [n:127.0.0.1:47805_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=deleteNode_collection&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1612016 INFO (OverseerThreadFactory-3795-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection deleteNode_collection
[junit4] 2> 1612127 INFO (OverseerStateUpdate-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deleteNode_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deleteNode_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:57779/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1612130 INFO (OverseerStateUpdate-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deleteNode_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deleteNode_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39954/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1612356 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr x:deleteNode_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteNode_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1612369 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr x:deleteNode_collection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteNode_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1613422 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 1613423 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 1613431 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.s.IndexSchema [deleteNode_collection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1613431 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.s.IndexSchema [deleteNode_collection_shard1_replica_n2] Schema name=minimal
[junit4] 2> 1613435 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1613435 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1613435 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'deleteNode_collection_shard1_replica_n2' using configuration from collection deleteNode_collection, trusted=true
[junit4] 2> 1613435 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'deleteNode_collection_shard1_replica_n1' using configuration from collection deleteNode_collection, trusted=true
[junit4] 2> 1613435 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39954.solr.core.deleteNode_collection.shard1.replica_n2' (registry 'solr.core.deleteNode_collection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1613435 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57779.solr.core.deleteNode_collection.shard1.replica_n1' (registry 'solr.core.deleteNode_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1613435 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1613435 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1613436 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.SolrCore [[deleteNode_collection_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node3/deleteNode_collection_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node3/./deleteNode_collection_shard1_replica_n2/data/]
[junit4] 2> 1613436 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SolrCore [[deleteNode_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node4/deleteNode_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node4/./deleteNode_collection_shard1_replica_n1/data/]
[junit4] 2> 1613516 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1613516 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1613516 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1613516 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1613518 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1613518 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1613518 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1613518 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1613520 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1895a82f[deleteNode_collection_shard1_replica_n2] main]
[junit4] 2> 1613520 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@311ae89a[deleteNode_collection_shard1_replica_n1] main]
[junit4] 2> 1613521 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1613522 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1613522 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1613522 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1613523 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1613523 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1613523 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600364025657425920
[junit4] 2> 1613524 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600364025658474496
[junit4] 2> 1613526 INFO (searcherExecutor-3806-thread-1-processing-n:127.0.0.1:39954_solr x:deleteNode_collection_shard1_replica_n2 c:deleteNode_collection s:shard1 r:core_node4) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.SolrCore [deleteNode_collection_shard1_replica_n2] Registered new searcher Searcher@1895a82f[deleteNode_collection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1613527 INFO (searcherExecutor-3807-thread-1-processing-n:127.0.0.1:57779_solr x:deleteNode_collection_shard1_replica_n1 c:deleteNode_collection s:shard1 r:core_node3) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SolrCore [deleteNode_collection_shard1_replica_n1] Registered new searcher Searcher@311ae89a[deleteNode_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1613535 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteNode_collection/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 1613538 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteNode_collection/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
[junit4] 2> 1613542 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1613542 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1613543 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:57779/solr/deleteNode_collection_shard1_replica_n1/
[junit4] 2> 1613543 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=deleteNode_collection_shard1_replica_n1 url=https://127.0.0.1:57779/solr START replicas=[https://127.0.0.1:39954/solr/deleteNode_collection_shard1_replica_n2/] nUpdates=100
[junit4] 2> 1613547 INFO (qtp1832386451-10159) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.c.S.Request [deleteNode_collection_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1613844 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=deleteNode_collection_shard1_replica_n1 url=https://127.0.0.1:57779/solr DONE. We have no versions. sync failed.
[junit4] 2> 1613844 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1613844 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1613844 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1613852 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:57779/solr/deleteNode_collection_shard1_replica_n1/ shard1
[junit4] 2> 1613969 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1613974 INFO (qtp571615543-10132) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteNode_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1617
[junit4] 2> 1614072 INFO (zkCallback-2888-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1614545 INFO (qtp1832386451-10158) [n:127.0.0.1:39954_solr c:deleteNode_collection s:shard1 r:core_node4 x:deleteNode_collection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteNode_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2176
[junit4] 2> 1614549 INFO (qtp1758755827-10121) [n:127.0.0.1:47805_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1614644 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1614644 INFO (zkCallback-2888-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1615551 INFO (qtp1758755827-10121) [n:127.0.0.1:47805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=deleteNode_collection&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3546
[junit4] 2> 1615552 INFO (qtp1758755827-10146) [n:127.0.0.1:47805_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=deleteNode_collection&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1615555 INFO (OverseerCollectionConfigSetProcessor-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1615558 INFO (OverseerThreadFactory-3795-thread-2) [ c:deleteNode_collection s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:47805_solr for creating new replica
[junit4] 2> 1615563 INFO (OverseerStateUpdate-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"deleteNode_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deleteNode_collection_shard1_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:47805/solr",
[junit4] 2> "node_name":"127.0.0.1:47805_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1615665 INFO (zkCallback-2888-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1615665 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1615768 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&name=deleteNode_collection_shard1_replica_n5&action=CREATE&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1615769 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1615782 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 1615790 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.s.IndexSchema [deleteNode_collection_shard1_replica_n5] Schema name=minimal
[junit4] 2> 1615793 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1615793 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'deleteNode_collection_shard1_replica_n5' using configuration from collection deleteNode_collection, trusted=true
[junit4] 2> 1615794 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47805.solr.core.deleteNode_collection.shard1.replica_n5' (registry 'solr.core.deleteNode_collection.shard1.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1615794 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1615794 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.SolrCore [[deleteNode_collection_shard1_replica_n5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node2/deleteNode_collection_shard1_replica_n5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node2/./deleteNode_collection_shard1_replica_n5/data/]
[junit4] 2> 1615879 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1615879 INFO (zkCallback-2888-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1615889 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1615889 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1615891 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1615891 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1615893 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@38bcf3ed[deleteNode_collection_shard1_replica_n5] main]
[junit4] 2> 1615895 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1615895 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1615896 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1615897 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600364028146745344
[junit4] 2> 1615901 INFO (searcherExecutor-3816-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.SolrCore [deleteNode_collection_shard1_replica_n5] Registered new searcher Searcher@38bcf3ed[deleteNode_collection_shard1_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1615904 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteNode_collection/terms/shard1 to Terms{values={core_node6=0, core_node3=0, core_node4=0}, version=2}
[junit4] 2> 1615906 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.ZkController Core needs to recover:deleteNode_collection_shard1_replica_n5
[junit4] 2> 1615907 INFO (updateExecutor-2870-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1615913 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1615914 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1615914 INFO (qtp1758755827-10145) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&name=deleteNode_collection_shard1_replica_n5&action=CREATE&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=145
[junit4] 2> 1615917 INFO (qtp1758755827-10146) [n:127.0.0.1:47805_solr c:deleteNode_collection ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=ADDREPLICA&collection=deleteNode_collection&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=364
[junit4] 2> 1615918 INFO (qtp1758755827-10147) [n:127.0.0.1:47805_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=deleteNode_collection&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1615921 INFO (qtp571615543-10136) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.S.Request [deleteNode_collection_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1615921 INFO (qtp571615543-10136) [n:127.0.0.1:57779_solr c:deleteNode_collection s:shard1 r:core_node3 x:deleteNode_collection_shard1_replica_n1] o.a.s.c.S.Request [deleteNode_collection_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1615921 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[deleteNode_collection_shard1_replica_n5]
[junit4] 2> 1615921 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1615921 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.RecoveryStrategy Publishing state of core [deleteNode_collection_shard1_replica_n5] as recovering, leader is [https://127.0.0.1:57779/solr/deleteNode_collection_shard1_replica_n1/] and I am [https://127.0.0.1:47805/solr/deleteNode_collection_shard1_replica_n5/]
[junit4] 2> 1615925 INFO (OverseerCollectionConfigSetProcessor-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1615933 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteNode_collection/terms/shard1 to Terms{values={core_node6=0, core_node6_recovering=0, core_node3=0, core_node4=0}, version=3}
[junit4] 2> 1615936 INFO (recoveryExecutor-2879-thread-1-processing-n:127.0.0.1:47805_solr x:deleteNode_collection_shard1_replica_n5 c:deleteNode_collection s:shard1 r:core_node6) [n:127.0.0.1:47805_solr c:deleteNode_collection s:shard1 r:core_node6 x:deleteNode_collection_shard1_replica_n5] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:57779/solr]; [WaitForState: action=PREPRECOVERY&core=deleteNode_collection_shard1_replica_n1&nodeName=127.0.0.1:47805_solr&coreNodeName=core_node6&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1615936 INFO (OverseerThreadFactory-3795-thread-3) [ c:deleteNode_collection s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:54181_solr for creating new replica
[junit4] 2> 1615939 INFO (OverseerStateUpdate-73792853437382669-127.0.0.1:57779_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"deleteNode_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deleteNode_collection_shard1_replica_t7",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54181/solr",
[junit4] 2> "node_name":"127.0.0.1:54181_solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 1615948 INFO (qtp571615543-10138) [n:127.0.0.1:57779_solr x:deleteNode_collection_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node6, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 1615950 INFO (qtp571615543-10138) [n:127.0.0.1:57779_solr x:deleteNode_collection_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=deleteNode_collection, shard=shard1, thisCore=deleteNode_collection_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:47805_solr, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{"core":"deleteNode_collection_shard1_replica_n5","base_url":"https://127.0.0.1:47805/solr","node_name":"127.0.0.1:47805_solr","state":"down","type":"NRT","force_set_state":"false"}
[junit4] 2> 1616044 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616044 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616045 INFO (zkCallback-2888-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616048 INFO (zkCallback-2886-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616149 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&name=deleteNode_collection_shard1_replica_t7&action=CREATE&collection=deleteNode_collection&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1616162 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 1616170 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.s.IndexSchema [deleteNode_collection_shard1_replica_t7] Schema name=minimal
[junit4] 2> 1616174 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1616174 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.CoreContainer Creating SolrCore 'deleteNode_collection_shard1_replica_t7' using configuration from collection deleteNode_collection, trusted=true
[junit4] 2> 1616177 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54181.solr.core.deleteNode_collection.shard1.replica_t7' (registry 'solr.core.deleteNode_collection.shard1.replica_t7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 1616178 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1616178 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.SolrCore [[deleteNode_collection_shard1_replica_t7] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node1/deleteNode_collection_shard1_replica_t7], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_5E14A061AE531331-001/tempDir-001/node1/./deleteNode_collection_shard1_replica_t7/data/]
[junit4] 2> 1616250 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1616250 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1616252 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1616252 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1616254 INFO (zkCallback-2888-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616254 INFO (zkCallback-2886-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616254 INFO (zkCallback-2886-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616254 INFO (zkCallback-2884-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteNode_collection/state.json] for collection [deleteNode_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1616256 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a975371[deleteNode_collection_shard1_replica_t7] main]
[junit4] 2> 1616258 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1616258 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1616259 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1616260 INFO (searcherExecutor-3821-thread-1-processing-n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7 c:deleteNode_collection s:shard1 r:core_node8) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.SolrCore [deleteNode_collection_shard1_replica_t7] Registered new searcher Searcher@7a975371[deleteNode_collection_shard1_replica_t7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1616260 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600364028527378432
[junit4] 2> 1616267 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteNode_collection/terms/shard1 to Terms{values={core_node6=0, core_node8=0, core_node6_recovering=0, core_node3=0, core_node4=0}, version=4}
[junit4] 2> 1616269 INFO (qtp277687652-10127) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.ZkController Core needs to recover:deleteNode_collection_shard1_replica_t7
[junit4] 2> 1616270 INFO (updateExecutor-2862-thread-1-processing-n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7 c:deleteNode_collection s:shard1 r:core_node8) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1616270 INFO (recoveryExecutor-2867-thread-1-processing-n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7 c:deleteNode_collection s:shard1 r:core_node8) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1616273 INFO (recoveryExecutor-2867-thread-1-processing-n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7 c:deleteNode_collection s:shard1 r:core_node8) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1616273 INFO (recoveryExecutor-2867-thread-1-processing-n:127.0.0.1:54181_solr x:deleteNode_collection_shard1_replica_t7 c:deleteNode_collection s:shard1 r:core_node8) [n:127.0.0.1:54181_solr c:deleteNode_collection s:shard1 r:core_node8 x:deleteNode_collection_shard1_replica_t7] o.a.s.c.ZkController deleteNode_collection_shard1_replica_t7 stopping background replication from leader
[junit4] 2> 1616275 INFO (qtp277687652-10
[...truncated too long message...]
lowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null370765764
[copy] Copying 40 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null370765764
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null370765764
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1999070075
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1999070075
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1999070075
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 76ms :: artifacts dl 6ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 164 minutes 8 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 67 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/67/
1 tests failed.
FAILED: org.apache.solr.cloud.TestAuthenticationFramework.testBasics
Error Message:
Error from server at https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([970938951E7E6CC1:AAD196B9269032B1]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:202)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Build Log:
[...truncated 14236 lines...]
[junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/init-core-data-001
[junit4] 2> 2964077 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001
[junit4] 2> 2964078 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2964078 INFO (Thread-5301) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2964078 INFO (Thread-5301) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2964083 ERROR (Thread-5301) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2964178 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.ZkTestServer start zk server on port:59101
[junit4] 2> 2964198 INFO (zkConnectionManagerCallback-5200-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964213 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2964221 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2964226 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2964226 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2964226 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2964227 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7d36a0d3{/solr,null,AVAILABLE}
[junit4] 2> 2964227 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4ba6d803{SSL,[ssl, http/1.1]}{127.0.0.1:41228}
[junit4] 2> 2964227 INFO (jetty-launcher-5197-thread-2) [ ] o.e.j.s.Server Started @2964283ms
[junit4] 2> 2964227 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41228}
[junit4] 2> 2964227 ERROR (jetty-launcher-5197-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2964228 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2964228 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 2964228 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2964228 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2964228 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-26T04:39:10.433Z
[junit4] 2> 2964233 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2964233 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2964233 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2964233 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49d23400{/solr,null,AVAILABLE}
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4dcd7982{SSL,[ssl, http/1.1]}{127.0.0.1:37910}
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.e.j.s.Server Started @2964290ms
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37910}
[junit4] 2> 2964234 ERROR (jetty-launcher-5197-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2964234 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-26T04:39:10.439Z
[junit4] 2> 2964237 INFO (zkConnectionManagerCallback-5204-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964237 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2964245 INFO (zkConnectionManagerCallback-5202-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964246 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2964255 INFO (jetty-launcher-5197-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59101/solr
[junit4] 2> 2964257 INFO (jetty-launcher-5197-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59101/solr
[junit4] 2> 2964260 INFO (zkConnectionManagerCallback-5212-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964261 INFO (zkConnectionManagerCallback-5208-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964264 INFO (zkConnectionManagerCallback-5214-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964278 INFO (zkConnectionManagerCallback-5216-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964300 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2964301 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41228_solr
[junit4] 2> 2964302 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.Overseer Overseer (id=73863861049819141-127.0.0.1:41228_solr-n_0000000000) starting
[junit4] 2> 2964342 INFO (zkConnectionManagerCallback-5225-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964345 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59101/solr ready
[junit4] 2> 2964369 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2964373 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37910_solr
[junit4] 2> 2964375 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2964375 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41228_solr
[junit4] 2> 2964400 INFO (zkCallback-5213-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2964410 INFO (zkCallback-5224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2964427 INFO (zkCallback-5215-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2964552 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
[junit4] 2> 2964614 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37910.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964625 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37910.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964626 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37910.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964627 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node1/.
[junit4] 2> 2964631 INFO (zkConnectionManagerCallback-5230-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964632 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2964633 INFO (jetty-launcher-5197-thread-1) [n:127.0.0.1:37910_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59101/solr ready
[junit4] 2> 2964809 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
[junit4] 2> 2964848 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41228.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964860 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41228.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964860 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41228.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2964861 INFO (jetty-launcher-5197-thread-2) [n:127.0.0.1:41228_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node2/.
[junit4] 2> 2964893 INFO (zkConnectionManagerCallback-5232-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964897 INFO (zkConnectionManagerCallback-5237-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2964898 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2964899 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59101/solr ready
[junit4] 2> 2964933 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2964935 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2964939 INFO (qtp197383304-22137) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2964940 INFO (qtp197383304-22137) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2964941 INFO (OverseerThreadFactory-8064-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
[junit4] 2> 2965049 INFO (OverseerStateUpdate-73863861049819141-127.0.0.1:41228_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41228/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2965052 INFO (OverseerStateUpdate-73863861049819141-127.0.0.1:41228_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37910/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2965055 INFO (OverseerStateUpdate-73863861049819141-127.0.0.1:41228_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41228/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2965058 INFO (OverseerStateUpdate-73863861049819141-127.0.0.1:41228_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37910/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2965272 INFO (qtp197383304-22137) [n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
[junit4] 2> 2965273 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 2965273 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2965276 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=4
[junit4] 2> 2965281 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 2965324 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
[junit4] 2> 2965325 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 2965327 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=1
[junit4] 2> 2965328 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 2965337 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048
[junit4] 2> 2965337 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2965941 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2965942 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2966309 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2966309 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2966317 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2966317 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema [testcollection_shard2_replica_n6] Schema name=minimal
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n6' using configuration from collection testcollection, trusted=true
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37910.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37910.solr.core.testcollection.shard2.replica_n6' (registry 'solr.core.testcollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node1/testcollection_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node1/./testcollection_shard1_replica_n2/data/]
[junit4] 2> 2966321 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [[testcollection_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node1/testcollection_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node1/./testcollection_shard2_replica_n6/data/]
[junit4] 2> 2966343 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048
[junit4] 2> 2966343 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2966347 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2966380 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2966396 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
[junit4] 2> 2966399 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2966399 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
[junit4] 2> 2966400 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2966407 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41228.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2966407 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2966407 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node2/testcollection_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node2/./testcollection_shard2_replica_n4/data/]
[junit4] 2> 2966411 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2966411 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
[junit4] 2> 2966411 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41228.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3af45ffb
[junit4] 2> 2966411 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2966411 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_970938951E7E6CC1-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
[junit4] 2> 2966464 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2966464 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2966465 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2966465 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2966474 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@18d23f45[testcollection_shard1_replica_n2] main]
[junit4] 2> 2966475 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2966476 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2966476 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2966477 INFO (searcherExecutor-8070-thread-1-processing-x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 r:core_node5 CREATE n:127.0.0.1:37910_solr c:testcollection s:shard1) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@18d23f45[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2966478 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601500141514129408
[junit4] 2> 2966481 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2966481 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2966483 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2966483 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2966484 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@2301ade3[testcollection_shard2_replica_n6] main]
[junit4] 2> 2966486 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2966486 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node5=0}, version=0}
[junit4] 2> 2966486 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2966487 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2966487 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601500141523566592
[junit4] 2> 2966489 INFO (searcherExecutor-8069-thread-1-processing-x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 r:core_node8 CREATE n:127.0.0.1:37910_solr c:testcollection s:shard2) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] Registered new searcher Searcher@2301ade3[testcollection_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2966491 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2966493 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node8=0}, version=0}
[junit4] 2> 2966500 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2966543 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2966543 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2966543 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2966543 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2966544 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2966544 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2966544 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2966544 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2966546 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a25033f[testcollection_shard2_replica_n4] main]
[junit4] 2> 2966546 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15076a25[testcollection_shard1_replica_n1] main]
[junit4] 2> 2966547 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2966548 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2966549 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2966549 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601500141588578304
[junit4] 2> 2966550 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2966551 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2966552 INFO (searcherExecutor-8072-thread-1-processing-x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 r:core_node3 CREATE n:127.0.0.1:41228_solr c:testcollection s:shard1) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@15076a25[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2966554 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2966555 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601500141594869760
[junit4] 2> 2966556 INFO (searcherExecutor-8071-thread-1-processing-x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 r:core_node7 CREATE n:127.0.0.1:41228_solr c:testcollection s:shard2) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@2a25033f[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2966559 INFO (parallelCoreAdminExecutor-8067-thread-2-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448294744833 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
[junit4] 2> 2966563 INFO (parallelCoreAdminExecutor-8067-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard2_replica_n4 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187 CREATE) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
[junit4] 2> 2966943 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2966944 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2966992 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2966992 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2966992 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/
[junit4] 2> 2966993 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=https://127.0.0.1:37910/solr START replicas=[https://127.0.0.1:41228/solr/testcollection_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2967002 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2967003 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2967003 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2967003 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37910/solr/testcollection_shard2_replica_n6/
[junit4] 2> 2967004 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=https://127.0.0.1:37910/solr START replicas=[https://127.0.0.1:41228/solr/testcollection_shard2_replica_n4/] nUpdates=100
[junit4] 2> 2967017 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2967293 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=https://127.0.0.1:37910/solr DONE. We have no versions. sync failed.
[junit4] 2> 2967294 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2967294 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2967294 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2967298 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/ shard1
[junit4] 2> 2967305 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=https://127.0.0.1:37910/solr DONE. We have no versions. sync failed.
[junit4] 2> 2967305 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2967305 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2967305 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2967308 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37910/solr/testcollection_shard2_replica_n6/ shard2
[junit4] 2> 2967344 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048
[junit4] 2> 2967344 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2967411 INFO (parallelCoreAdminExecutor-8065-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard2_replica_n6 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2967450 INFO (parallelCoreAdminExecutor-8065-thread-2-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448297703003 CREATE) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2967556 INFO (zkCallback-5215-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2967676 INFO (zkCallback-5215-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2967676 INFO (zkCallback-5213-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2967945 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2967946 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2968345 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048
[junit4] 2> 2968345 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448303542048&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2968348 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187
[junit4] 2> 2968348 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0ae7e8fb-c927-4cd6-b12a-b53dd265504527561448300637187&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2968350 INFO (OverseerThreadFactory-8064-thread-1) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2968951 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2968952 INFO (OverseerCollectionConfigSetProcessor-73863861049819141-127.0.0.1:41228_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2968953 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2968954 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2968955 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0ae7e8fb-c927-4cd6-b12a-b53dd2655045&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2968955 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 1> replica:core_node3 rstate:active live:true
[junit4] 1> replica:core_node5 rstate:active live:true
[junit4] 1> replica:core_node7 rstate:active live:true
[junit4] 1> replica:core_node8 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 2968956 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[970938951E7E6CC1]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testcollection
[junit4] 2> 2968963 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:6&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:41228/solr/testcollection_shard1_replica_n1/|https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&rows=10&version=2&q=*:*&NOW=1527309555161&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2968963 INFO (qtp1032987856-22126) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:6&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:41228/solr/testcollection_shard2_replica_n4/|https://127.0.0.1:37910/solr/testcollection_shard2_replica_n6/&rows=10&version=2&q=*:*&NOW=1527309555161&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2968964 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.S.Request [testcollection_shard2_replica_n6] webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:6&wt=javabin&version=2} hits=0 status=0 QTime=7
[junit4] 2> 2969013 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=1, core_node5=1}, version=2}
[junit4] 2> 2969049 INFO (qtp1032987856-22126) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&wt=javabin&version=2}{add=[1 (1601500144127180800)]} 0 26
[junit4] 2> 2969049 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={_stateVer_=testcollection:6&wt=javabin&version=2}{add=[1 (1601500144127180800)]} 0 79
[junit4] 2> 2969053 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601500144214212608,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2969053 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@785b0a88 commitCommandVersion:1601500144214212608
[junit4] 2> 2969112 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601500144276078592,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2969112 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7733be1 commitCommandVersion:1601500144276078592
[junit4] 2> 2969117 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601500144281321472,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2969124 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601500144288661504,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2969124 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2969124 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2969126 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@785bfc6e[testcollection_shard1_replica_n1] main]
[junit4] 2> 2969127 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2969128 INFO (searcherExecutor-8072-thread-1-processing-n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1 c:testcollection s:shard1 r:core_node3) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@785bfc6e[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.4.0):C1)))}
[junit4] 2> 2969129 INFO (qtp1032987856-22128) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 75
[junit4] 2> 2969139 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2969139 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n4] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 15
[junit4] 2> 2969139 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2969139 INFO (qtp197383304-22135) [n:127.0.0.1:37910_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 22
[junit4] 2> 2969154 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@27418c12[testcollection_shard1_replica_n2] main]
[junit4] 2> 2969154 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2969156 INFO (searcherExecutor-8070-thread-1-processing-n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2 c:testcollection s:shard1 r:core_node5) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@27418c12[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.4.0):C1)))}
[junit4] 2> 2969157 INFO (qtp197383304-22132) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 45
[junit4] 2> 2969157 INFO (qtp197383304-22138) [n:127.0.0.1:37910_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={_stateVer_=testcollection:6&commit=true&wt=javabin&version=2}{commit=} 0 104
[junit4] 2> 2969174 INFO (qtp1032987856-22129) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:6&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:41228/solr/testcollection_shard1_replica_n1/|https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&rows=10&version=2&q=*:*&NOW=1527309555365&isShard=true&wt=javabin} hits=1 status=0 QTime=0
[junit4] 2> 2969176 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:6&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:41228/solr/testcollection_shard2_replica_n4/|https://127.0.0.1:37910/solr/testcollection_shard2_replica_n6/&rows=10&version=2&q=*:*&NOW=1527309555365&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2969182 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=testcollection:6&shards.purpose=64&NOW=1527309555365&ids=1&isShard=true&shard.url=https://127.0.0.1:41228/solr/testcollection_shard1_replica_n1/|https://127.0.0.1:37910/solr/testcollection_shard1_replica_n2/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2969183 INFO (qtp1032987856-22126) [n:127.0.0.1:41228_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:6&wt=javabin&version=2} hits=1 status=0 QTime=23
[junit4] 2> 2969184 INFO (qtp197383304-22137) [n:127.0.0.1:37910_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=testcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2969211 INFO (OverseerThreadFactory-8064-thread-2) [ ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
[junit4] 2> 2969218 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=null
[junit4] 2> 2969218 INFO (qtp1032987856-22125) [n:127.0.0.1:41228_solr x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@51f91d1b: rootName = solr_41228, domain = solr.core.testcollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@39c7d511
[junit4] 2> 2969240 INFO (qtp197383304-22136) [n:127.0.0.1:37910_solr x:testcollection_shard1_replica_n2] o.a.s.m.SolrMet
[...truncated too long message...]
d-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1793525358
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1793525358
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1793525358
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1562408350
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1562408350
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1562408350
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 54ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 163 minutes 17 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 66 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/66/
3 tests failed.
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node4:{"core":"c8n_1x3_lf_shard1_replica_n3","base_url":"http://127.0.0.1:44135","node_name":"127.0.0.1:44135_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/15)={ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node2":{ "core":"c8n_1x3_lf_shard1_replica_n1", "base_url":"http://127.0.0.1:37443", "node_name":"127.0.0.1:37443_", "state":"down", "type":"NRT"}, "core_node4":{ "core":"c8n_1x3_lf_shard1_replica_n3", "base_url":"http://127.0.0.1:44135", "node_name":"127.0.0.1:44135_", "state":"active", "type":"NRT", "leader":"true"}, "core_node6":{ "core":"c8n_1x3_lf_shard1_replica_n5", "base_url":"http://127.0.0.1:43870", "node_name":"127.0.0.1:43870_", "state":"down", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node4:{"core":"c8n_1x3_lf_shard1_replica_n3","base_url":"http://127.0.0.1:44135","node_name":"127.0.0.1:44135_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/15)={
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node2":{
"core":"c8n_1x3_lf_shard1_replica_n1",
"base_url":"http://127.0.0.1:37443",
"node_name":"127.0.0.1:37443_",
"state":"down",
"type":"NRT"},
"core_node4":{
"core":"c8n_1x3_lf_shard1_replica_n3",
"base_url":"http://127.0.0.1:44135",
"node_name":"127.0.0.1:44135_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node6":{
"core":"c8n_1x3_lf_shard1_replica_n5",
"base_url":"http://127.0.0.1:43870",
"node_name":"127.0.0.1:43870_",
"state":"down",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([81E591950FFF0E5C:9B1AE4FA10363A4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
Error Message:
waitFor not elapsed but produced an event
Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
at __randomizedtesting.SeedInfo.seed([81E591950FFF0E5C:E22EA71796307D71]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap
Error Message:
Error from server at https://127.0.0.1:45544/solr: KeeperErrorCode = Session expired for /configs/cdcr-source
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45544/solr: KeeperErrorCode = Session expired for /configs/cdcr-source
at __randomizedtesting.SeedInfo.seed([81E591950FFF0E5C:5632BEE2BBA0961B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.cdcr.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap(CdcrBootstrapTest.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13575 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_81E591950FFF0E5C-001/init-core-data-001
[junit4] 2> 3365046 WARN (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=19 numCloses=19
[junit4] 2> 3365046 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3365047 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3365048 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3365049 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_81E591950FFF0E5C-001/tempDir-001
[junit4] 2> 3365049 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3365061 INFO (Thread-3942) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3365061 INFO (Thread-3942) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3365087 ERROR (Thread-3942) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3365149 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:33565
[junit4] 2> 3365164 INFO (zkConnectionManagerCallback-6244-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365177 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 3365183 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3365183 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3365183 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 3365183 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@303fb5c1{/solr,null,AVAILABLE}
[junit4] 2> 3365184 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7dd788fd{SSL,[ssl, http/1.1]}{127.0.0.1:43848}
[junit4] 2> 3365184 INFO (jetty-launcher-6241-thread-2) [ ] o.e.j.s.Server Started @3365381ms
[junit4] 2> 3365184 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43848}
[junit4] 2> 3365185 ERROR (jetty-launcher-6241-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3365185 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3365185 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 3365185 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3365185 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3365185 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-24T14:43:25.369Z
[junit4] 2> 3365197 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 3365219 INFO (zkConnectionManagerCallback-6246-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365219 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3365244 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3365244 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3365244 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 3365244 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51cad428{/solr,null,AVAILABLE}
[junit4] 2> 3365245 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7f282e4a{SSL,[ssl, http/1.1]}{127.0.0.1:43588}
[junit4] 2> 3365245 INFO (jetty-launcher-6241-thread-1) [ ] o.e.j.s.Server Started @3365441ms
[junit4] 2> 3365245 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43588}
[junit4] 2> 3365245 INFO (jetty-launcher-6241-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33565/solr
[junit4] 2> 3365245 ERROR (jetty-launcher-6241-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3365245 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 3365248 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 3365248 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3365248 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3365248 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-24T14:43:25.432Z
[junit4] 2> 3365249 INFO (zkConnectionManagerCallback-6250-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365283 INFO (zkConnectionManagerCallback-6252-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365284 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3365297 INFO (zkConnectionManagerCallback-6254-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365317 INFO (jetty-launcher-6241-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33565/solr
[junit4] 2> 3365334 INFO (zkConnectionManagerCallback-6260-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365335 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10236dd482a0005, likely client has closed socket
[junit4] 2> 3365354 INFO (zkConnectionManagerCallback-6262-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365421 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3365421 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43588_solr
[junit4] 2> 3365422 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.Overseer Overseer (id=72680868017733638-127.0.0.1:43588_solr-n_0000000000) starting
[junit4] 2> 3365451 INFO (zkConnectionManagerCallback-6269-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3365469 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33565/solr ready
[junit4] 2> 3365494 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3365494 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43588_solr
[junit4] 2> 3365495 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3365496 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 3365496 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3365526 INFO (zkCallback-6268-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3365526 INFO (zkCallback-6261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3365544 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:43588_solr]
[junit4] 2> 3365554 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3365557 DEBUG (OverseerAutoScalingTriggerThread-72680868017733638-127.0.0.1:43588_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3365557 DEBUG (ScheduledTrigger-4854-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3365695 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3365709 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3365711 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3365711 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43848_solr
[junit4] 2> 3365737 INFO (zkCallback-6268-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3365774 INFO (zkCallback-6261-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3365819 INFO (zkCallback-6253-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3366462 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43588.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366489 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43588.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366490 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43588.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366491 INFO (jetty-launcher-6241-thread-1) [n:127.0.0.1:43588_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_81E591950FFF0E5C-001/tempDir-001/node1/.
[junit4] 2> 3366595 DEBUG (ScheduledTrigger-4854-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366658 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43848.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366689 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43848.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366689 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43848.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c910b49
[junit4] 2> 3366690 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_81E591950FFF0E5C-001/tempDir-001/node2/.
[junit4] 2> 3366706 INFO (zkConnectionManagerCallback-6274-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3366707 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3366708 INFO (jetty-launcher-6241-thread-2) [n:127.0.0.1:43848_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33565/solr ready
[junit4] 2> 3366761 INFO (zkConnectionManagerCallback-6276-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3366763 INFO (zkConnectionManagerCallback-6281-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3366764 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3366765 INFO (SUITE-IndexSizeTriggerTest-seed#[81E591950FFF0E5C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33565/solr ready
[junit4] 2> 3366786 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3366786 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 3366786 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 3366786 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 3366787 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10134_solr, 127.0.0.1:10135_solr]
[junit4] 2> 3366787 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3366787 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 3366787 DEBUG (ScheduledTrigger-4863-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366808 DEBUG (ScheduledTrigger-4863-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366829 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366849 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366869 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366889 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366909 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366930 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366950 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366970 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3366991 DEBUG (ScheduledTrigger-4863-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367018 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367051 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367079 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367116 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367169 DEBUG (ScheduledTrigger-4863-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367173 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMixedBounds
[junit4] 2> 3367185 WARN (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.IndexSizeTriggerTest Requires SOLR-12208
[junit4] 2> 3367206 DEBUG (ScheduledTrigger-4863-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367206 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 3367207 DEBUG (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3367209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3367209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3367210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3367210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3367312 INFO (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMixedBounds
[junit4] 2> 3367548 INFO (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMergeIntegration
[junit4] 2> 3367605 DEBUG (ScheduledTrigger-4854-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3367620 DEBUG (simCloudManagerPool-4862-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testMergeIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 3367620 DEBUG (simCloudManagerPool-4862-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testMergeIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testMergeIntegration_collection"}}
[junit4] 2> 3367801 DEBUG (simCloudManagerPool-4862-thread-19) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3367801 DEBUG (simCloudManagerPool-4862-thread-19) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3367820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3367820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3367820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3367820 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3367823 DEBUG (simCloudManagerPool-4862-thread-20) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3367873 DEBUG (simCloudManagerPool-4862-thread-20) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3367873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3367873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3367873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3367873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3367900 DEBUG (simCloudManagerPool-4862-thread-21) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3367900 DEBUG (simCloudManagerPool-4862-thread-21) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3367917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3367917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3367917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3367917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3368101 DEBUG (ScheduledTrigger-4863-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 3368102 DEBUG (ScheduledTrigger-4863-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger]
[junit4] 2> 3368103 INFO (ScheduledTrigger-4863-thread-2) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646831050838628, stage=STARTED, actionName='null', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368113 DEBUG (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 3368114 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646831612048478, stage=BEFORE_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368114 DEBUG (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 3368115 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed plan empty, remained 0 requested ops to try.
[junit4] 2> 3368115 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 9646831659 sessionWrapper.createTime 9646831650749228, this.sessionWrapper.createTime 9646831650749228
[junit4] 2> 3368115 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646831661556228, stage=AFTER_ACTION, actionName='compute_plan', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368115 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646831677429178, stage=BEFORE_ACTION, actionName='execute_plan', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368116 DEBUG (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}} with context properties: {AFTER_ACTION=[compute_plan], unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], BEFORE_ACTION=[compute_plan, execute_plan]}
[junit4] 2> 3368116 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}
[junit4] 2> 3368149 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646831714802128, stage=AFTER_ACTION, actionName='execute_plan', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={properties.unsupportedOps=[Op{action=MERGESHARDS, hints={COLL_SHARD=[{
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard2"}, {
[junit4] 2> "first":"testMergeIntegration_collection",
[junit4] 2> "second":"shard1"}]}}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368149 INFO (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9646833375359778, stage=SUCCEEDED, actionName='null', event={
[junit4] 2> "id":"2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9646817751295428,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "__start__":1,
[junit4] 2> "aboveSize":{},
[junit4] 2> "belowSize":{"testMergeIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testMergeIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":7,\n \"SEARCHER.searcher.deletedDocs\":5,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":2,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":2,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testMergeIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testMergeIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":3,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":3,\n \"violationType\":\"belowDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":3,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testMergeIntegration_collection\"}}"]},
[junit4] 2> "_enqueue_time_":9646831019567078,
[junit4] 2> "requestedOps":["Op{action=MERGESHARDS, hints={COLL_SHARD=[{\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard2\"}, {\n \"first\":\"testMergeIntegration_collection\",\n \"second\":\"shard1\"}]}}"]}}, context={}, config={
[junit4] 2> "trigger":"index_size_trigger",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":[
[junit4] 2> "compute_plan",
[junit4] 2> "execute_plan"]}, message='null'}
[junit4] 2> 3368151 INFO (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [testMergeIntegration_collection]
[junit4] 2> DocCollection(testMergeIntegration_collection//clusterstate.json/21)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testMergeIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":3,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":3},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testMergeIntegration_collection_shard2_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":3,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10135_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":3}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testMergeIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":7,
[junit4] 2> "SEARCHER.searcher.deletedDocs":5,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":2},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testMergeIntegration_collection_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":7,
[junit4] 2> "SEARCHER.searcher.deletedDocs":5,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10135_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":2}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 4
[junit4] 2> ## * testMergeIntegration_collection 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 18
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: index_size_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 3368151 DEBUG (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 3368153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 3368153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3368153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3368153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 3368169 DEBUG (AutoscalingActionExecutor-4864-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2809 ms for event id=2245bac94765c4Tb5sfqb9o18aysh6lkeoeqh16g
[junit4] 2> 3368259 INFO (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMergeIntegration
[junit4] 2> 3368642 DEBUG (ScheduledTrigger-4854-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3369650 DEBUG (ScheduledTrigger-4854-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3369703 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
[junit4] 2> 3369763 DEBUG (simCloudManagerPool-4862-thread-35) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testTrigger_collection"}}
[junit4] 2> 3369764 DEBUG (simCloudManagerPool-4862-thread-36) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testTrigger_collection"}}
[junit4] 2> 3369836 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.IndexSizeTriggerTest #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [testTrigger_collection]
[junit4] 2> DocCollection(testTrigger_collection//clusterstate.json/51)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testTrigger_collection_shard2_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":11,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10135_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":11}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testTrigger_collection_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":14,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10135_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":14}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 4
[junit4] 2> ## * testTrigger_collection 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - update 27
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: index_size_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 3369836 DEBUG (TEST-IndexSizeTriggerTest.testTrigger-seed#[81E591950FFF0E5C]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 3369836 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 3369836 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3369836 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3369836 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 3369937 INFO (TEST-IndexSizeTriggerTest.testTrigger-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testTrigger -Dtests.seed=81E591950FFF0E5C -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CR -Dtests.timezone=Pacific/Honolulu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.68s J0 | IndexSizeTriggerTest.testTrigger <<<
[junit4] > Throwable #1: java.lang.AssertionError: waitFor not elapsed but produced an event
[junit4] > at __randomizedtesting.SeedInfo.seed([81E591950FFF0E5C:E22EA71796307D71]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3370667 DEBUG (ScheduledTrigger-4854-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3371625 INFO (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[81E591950FFF0E5C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSplitIntegration
[junit4] 2> 3371667 DEBUG (ScheduledTrigger-4854-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3371680 DEBUG (simCloudManagerPool-4862-thread-68) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2: {"core_node3":{
[junit4] 2> "core":"testSplitIntegration_collection_shard2_replica_n3",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 3371680 DEBUG (simCloudManagerPool-4862-thread-69) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1: {"core_node1":{
[junit4] 2> "core":"testSplitIntegration_collection_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10134_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testSplitIntegration_collection"}}
[junit4] 2> 3371693 DEBUG (simCloudManagerPool-4862-thread-70) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3371693 DEBUG (simCloudManagerPool-4862-thread-70) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 3371693 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 3371694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3371694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3371694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 3371694 DEBUG (simCloudManagerPool-4862-thread-71) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3371694 DEBUG (simCloudManagerPool-4862-thread-71) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 3371695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 3371695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3371695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3371695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 3371695 DEBUG (simCloudManagerPool-4862-thread-72) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3371695 DEBUG (simCloudManagerPool-4862-thread-72) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 3371696 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 3371717 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3371717 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3371717 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 3371770 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"2245e7b0949ed4Tb5sfqb9o18aysh6lkeoeqh16k",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9647010610454228,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":14,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":14,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":14,\n \"__bytes__\":1,\n \"shard\":\"shard1\",\n \"collection\":\"testSplitIntegration_collection\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":11,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":11,\n \"violationType\":\"aboveDocs\",\n \"node_name\":\"127.0.0.1:10134_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"SEARCHER.searcher.numDocs\":11,\n \"__bytes__\":1,\n \"shard\":\"shard2\",\n \"collection\":\"testSplitIntegration_collection\"}}"]},
[junit4] 2> "belowSize":{},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard1\"}]}}",
[junit4] 2> "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n \"first\":\"testSplitIntegration_collection\",\n \"second\":\"shard2\"}]}}"]}}
[junit4] 2> 3371770 DEBUG (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger]
[junit4] 2> 3371770 INFO (ScheduledTrigger-4863-thread-4) [ ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=9647014409291328, stage=STARTED, actionName='null', event={
[junit4] 2> "id":"2245e7b0949ed4Tb5sfqb9o18aysh6lkeoeqh16k",
[junit4] 2> "source":"index_size_trigger",
[junit4] 2> "eventTime":9647010610454228,
[junit4] 2> "eventType":"INDEXSIZE",
[junit4] 2> "properties":{
[junit4] 2> "aboveSize":{"testSplitIntegration_collection":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n \"leader\":\"true\",\n \"SEARCHER.searcher.maxDoc\":14,\n \"SEARCHER.searcher.deletedDocs\":0,\n \"INDEX.sizeInBytes\":1,\n \"__docs__\":14,\n \"v
[...truncated too long message...]
ure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1941968067
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1941968067
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1941968067
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null288666196
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null288666196
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null288666196
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 57ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 316 minutes 23 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 65 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/65/
4 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
last state: DocCollection(testNodeLost//clusterstate.json/29)={ "replicationFactor":"10", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"5", "autoAddReplicas":"false", "nrtReplicas":"10", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard20":{ "replicas":{ "core_node191":{ "core":"testNodeLost_shard20_replica_n191", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node200":{ "core":"testNodeLost_shard20_replica_n200", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node199":{ "core":"testNodeLost_shard20_replica_n199", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node198":{ "core":"testNodeLost_shard20_replica_n198", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node197":{ "core":"testNodeLost_shard20_replica_n197", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node196":{ "core":"testNodeLost_shard20_replica_n196", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node195":{ "core":"testNodeLost_shard20_replica_n195", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node194":{ "core":"testNodeLost_shard20_replica_n194", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node193":{ "core":"testNodeLost_shard20_replica_n193", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node192":{ "core":"testNodeLost_shard20_replica_n192", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"73330000-7fffffff", "state":"active"}, "shard10":{ "replicas":{ "core_node92":{ "core":"testNodeLost_shard10_replica_n92", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node91":{ "core":"testNodeLost_shard10_replica_n91", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node94":{ "core":"testNodeLost_shard10_replica_n94", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node93":{ "core":"testNodeLost_shard10_replica_n93", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node96":{ "core":"testNodeLost_shard10_replica_n96", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node95":{ "core":"testNodeLost_shard10_replica_n95", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node100":{ "core":"testNodeLost_shard10_replica_n100", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node98":{ "core":"testNodeLost_shard10_replica_n98", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node97":{ "core":"testNodeLost_shard10_replica_n97", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node99":{ "core":"testNodeLost_shard10_replica_n99", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"f3330000-ffffffff", "state":"active"}, "shard11":{ "replicas":{ "core_node108":{ "core":"testNodeLost_shard11_replica_n108", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node107":{ "core":"testNodeLost_shard11_replica_n107", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node106":{ "core":"testNodeLost_shard11_replica_n106", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node105":{ "core":"testNodeLost_shard11_replica_n105", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node104":{ "core":"testNodeLost_shard11_replica_n104", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node103":{ "core":"testNodeLost_shard11_replica_n103", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node102":{ "core":"testNodeLost_shard11_replica_n102", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node101":{ "core":"testNodeLost_shard11_replica_n101", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node110":{ "core":"testNodeLost_shard11_replica_n110", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node109":{ "core":"testNodeLost_shard11_replica_n109", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"0-ccbffff", "state":"active"}, "shard12":{ "replicas":{ "core_node119":{ "core":"testNodeLost_shard12_replica_n119", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10022_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node118":{ "core":"testNodeLost_shard12_replica_n118", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node117":{ "core":"testNodeLost_shard12_replica_n117", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node116":{ "core":"testNodeLost_shard12_replica_n116", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node115":{ "core":"testNodeLost_shard12_replica_n115", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node114":{ "core":"testNodeLost_shard12_replica_n114", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node113":{ "core":"testNodeLost_shard12_replica_n113", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node112":{ "core":"testNodeLost_shard12_replica_n112", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node111":{ "core":"testNodeLost_shard12_replica_n111", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node120":{ "core":"testNodeLost_shard12_replica_n120", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"ccc0000-1998ffff", "state":"active"}, "shard13":{ "replicas":{ "core_node129":{ "core":"testNodeLost_shard13_replica_n129", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node128":{ "core":"testNodeLost_shard13_replica_n128", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node126":{ "core":"testNodeLost_shard13_replica_n126", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node125":{ "core":"testNodeLost_shard13_replica_n125", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node202":{ "node_name":"127.0.0.1:10072_solr", "core":"testNodeLost_shard13_replica_n201", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node124":{ "core":"testNodeLost_shard13_replica_n124", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node123":{ "core":"testNodeLost_shard13_replica_n123", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node122":{ "core":"testNodeLost_shard13_replica_n122", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node121":{ "core":"testNodeLost_shard13_replica_n121", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node130":{ "core":"testNodeLost_shard13_replica_n130", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10021_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"19990000-2665ffff", "state":"active"}, "shard2":{ "replicas":{ "core_node18":{ "core":"testNodeLost_shard2_replica_n18", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node17":{ "core":"testNodeLost_shard2_replica_n17", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10004_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node19":{ "core":"testNodeLost_shard2_replica_n19", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10022_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node20":{ "core":"testNodeLost_shard2_replica_n20", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node12":{ "core":"testNodeLost_shard2_replica_n12", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node11":{ "core":"testNodeLost_shard2_replica_n11", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node14":{ "core":"testNodeLost_shard2_replica_n14", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node13":{ "core":"testNodeLost_shard2_replica_n13", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node16":{ "core":"testNodeLost_shard2_replica_n16", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node15":{ "core":"testNodeLost_shard2_replica_n15", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"8ccc0000-9998ffff", "state":"active"}, "shard3":{ "replicas":{ "core_node29":{ "core":"testNodeLost_shard3_replica_n29", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node28":{ "core":"testNodeLost_shard3_replica_n28", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node204":{ "node_name":"127.0.0.1:10056_solr", "core":"testNodeLost_shard3_replica_n203", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node30":{ "core":"testNodeLost_shard3_replica_n30", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10021_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node21":{ "core":"testNodeLost_shard3_replica_n21", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node23":{ "core":"testNodeLost_shard3_replica_n23", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node22":{ "core":"testNodeLost_shard3_replica_n22", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node25":{ "core":"testNodeLost_shard3_replica_n25", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node24":{ "core":"testNodeLost_shard3_replica_n24", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node26":{ "core":"testNodeLost_shard3_replica_n26", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"99990000-a665ffff", "state":"active"}, "shard14":{ "replicas":{ "core_node139":{ "core":"testNodeLost_shard14_replica_n139", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node138":{ "core":"testNodeLost_shard14_replica_n138", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node137":{ "core":"testNodeLost_shard14_replica_n137", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node136":{ "core":"testNodeLost_shard14_replica_n136", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node135":{ "core":"testNodeLost_shard14_replica_n135", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node134":{ "core":"testNodeLost_shard14_replica_n134", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node133":{ "core":"testNodeLost_shard14_replica_n133", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node132":{ "core":"testNodeLost_shard14_replica_n132", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node131":{ "core":"testNodeLost_shard14_replica_n131", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node140":{ "core":"testNodeLost_shard14_replica_n140", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"26660000-3332ffff", "state":"active"}, "shard15":{ "replicas":{ "core_node149":{ "core":"testNodeLost_shard15_replica_n149", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node148":{ "core":"testNodeLost_shard15_replica_n148", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node147":{ "core":"testNodeLost_shard15_replica_n147", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node146":{ "core":"testNodeLost_shard15_replica_n146", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10005_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node145":{ "core":"testNodeLost_shard15_replica_n145", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node144":{ "core":"testNodeLost_shard15_replica_n144", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node143":{ "core":"testNodeLost_shard15_replica_n143", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node142":{ "core":"testNodeLost_shard15_replica_n142", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node141":{ "core":"testNodeLost_shard15_replica_n141", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node150":{ "core":"testNodeLost_shard15_replica_n150", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"33330000-3fffffff", "state":"active"}, "shard4":{ "replicas":{ "core_node39":{ "core":"testNodeLost_shard4_replica_n39", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node40":{ "core":"testNodeLost_shard4_replica_n40", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node32":{ "core":"testNodeLost_shard4_replica_n32", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node31":{ "core":"testNodeLost_shard4_replica_n31", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node34":{ "core":"testNodeLost_shard4_replica_n34", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node33":{ "core":"testNodeLost_shard4_replica_n33", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node36":{ "core":"testNodeLost_shard4_replica_n36", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node35":{ "core":"testNodeLost_shard4_replica_n35", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node38":{ "core":"testNodeLost_shard4_replica_n38", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node37":{ "core":"testNodeLost_shard4_replica_n37", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"a6660000-b332ffff", "state":"active"}, "shard5":{ "replicas":{ "core_node50":{ "core":"testNodeLost_shard5_replica_n50", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node41":{ "core":"testNodeLost_shard5_replica_n41", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node43":{ "core":"testNodeLost_shard5_replica_n43", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node42":{ "core":"testNodeLost_shard5_replica_n42", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node45":{ "core":"testNodeLost_shard5_replica_n45", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node44":{ "core":"testNodeLost_shard5_replica_n44", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node47":{ "core":"testNodeLost_shard5_replica_n47", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node46":{ "core":"testNodeLost_shard5_replica_n46", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10005_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node49":{ "core":"testNodeLost_shard5_replica_n49", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node48":{ "core":"testNodeLost_shard5_replica_n48", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"b3330000-bfffffff", "state":"active"}, "shard16":{ "replicas":{ "core_node159":{ "core":"testNodeLost_shard16_replica_n159", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node158":{ "core":"testNodeLost_shard16_replica_n158", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node157":{ "core":"testNodeLost_shard16_replica_n157", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node156":{ "core":"testNodeLost_shard16_replica_n156", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node155":{ "core":"testNodeLost_shard16_replica_n155", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10096_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node154":{ "core":"testNodeLost_shard16_replica_n154", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node153":{ "core":"testNodeLost_shard16_replica_n153", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node152":{ "core":"testNodeLost_shard16_replica_n152", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node151":{ "core":"testNodeLost_shard16_replica_n151", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node160":{ "core":"testNodeLost_shard16_replica_n160", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10049_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"40000000-4ccbffff", "state":"active"}, "shard17":{ "replicas":{ "core_node169":{ "core":"testNodeLost_shard17_replica_n169", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node168":{ "core":"testNodeLost_shard17_replica_n168", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node167":{ "core":"testNodeLost_shard17_replica_n167", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node166":{ "core":"testNodeLost_shard17_replica_n166", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node165":{ "core":"testNodeLost_shard17_replica_n165", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node164":{ "core":"testNodeLost_shard17_replica_n164", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node163":{ "core":"testNodeLost_shard17_replica_n163", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node162":{ "core":"testNodeLost_shard17_replica_n162", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10065_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node161":{ "core":"testNodeLost_shard17_replica_n161", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node170":{ "core":"testNodeLost_shard17_replica_n170", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"4ccc0000-5998ffff", "state":"active"}, "shard18":{ "replicas":{ "core_node180":{ "core":"testNodeLost_shard18_replica_n180", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node179":{ "core":"testNodeLost_shard18_replica_n179", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node178":{ "core":"testNodeLost_shard18_replica_n178", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node177":{ "core":"testNodeLost_shard18_replica_n177", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node176":{ "core":"testNodeLost_shard18_replica_n176", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node175":{ "core":"testNodeLost_shard18_replica_n175", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node174":{ "core":"testNodeLost_shard18_replica_n174", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node173":{ "core":"testNodeLost_shard18_replica_n173", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node172":{ "core":"testNodeLost_shard18_replica_n172", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node171":{ "core":"testNodeLost_shard18_replica_n171", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"59990000-6665ffff", "state":"active"}, "shard19":{ "replicas":{ "core_node190":{ "core":"testNodeLost_shard19_replica_n190", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node189":{ "core":"testNodeLost_shard19_replica_n189", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node188":{ "core":"testNodeLost_shard19_replica_n188", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node187":{ "core":"testNodeLost_shard19_replica_n187", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node186":{ "core":"testNodeLost_shard19_replica_n186", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node185":{ "core":"testNodeLost_shard19_replica_n185", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node184":{ "core":"testNodeLost_shard19_replica_n184", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node183":{ "core":"testNodeLost_shard19_replica_n183", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node182":{ "core":"testNodeLost_shard19_replica_n182", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node181":{ "core":"testNodeLost_shard19_replica_n181", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"66660000-7332ffff", "state":"active"}, "shard1":{ "replicas":{ "core_node1":{ "core":"testNodeLost_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node10":{ "core":"testNodeLost_shard1_replica_n10", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node6":{ "core":"testNodeLost_shard1_replica_n6", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node7":{ "core":"testNodeLost_shard1_replica_n7", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node8":{ "core":"testNodeLost_shard1_replica_n8", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node9":{ "core":"testNodeLost_shard1_replica_n9", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{ "core":"testNodeLost_shard1_replica_n2", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node3":{ "core":"testNodeLost_shard1_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testNodeLost_shard1_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node5":{ "core":"testNodeLost_shard1_replica_n5", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"80000000-8ccbffff", "state":"active"}, "shard6":{ "replicas":{ "core_node60":{ "core":"testNodeLost_shard6_replica_n60", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10049_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node52":{ "core":"testNodeLost_shard6_replica_n52", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node51":{ "core":"testNodeLost_shard6_replica_n51", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node54":{ "core":"testNodeLost_shard6_replica_n54", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node53":{ "core":"testNodeLost_shard6_replica_n53", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node56":{ "core":"testNodeLost_shard6_replica_n56", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node55":{ "core":"testNodeLost_shard6_replica_n55", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10096_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node58":{ "core":"testNodeLost_shard6_replica_n58", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node57":{ "core":"testNodeLost_shard6_replica_n57", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node59":{ "core":"testNodeLost_shard6_replica_n59", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"c0000000-cccbffff", "state":"active"}, "shard7":{ "replicas":{ "core_node70":{ "core":"testNodeLost_shard7_replica_n70", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node61":{ "core":"testNodeLost_shard7_replica_n61", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node63":{ "core":"testNodeLost_shard7_replica_n63", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node62":{ "core":"testNodeLost_shard7_replica_n62", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10065_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node65":{ "core":"testNodeLost_shard7_replica_n65", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node64":{ "core":"testNodeLost_shard7_replica_n64", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node67":{ "core":"testNodeLost_shard7_replica_n67", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node66":{ "core":"testNodeLost_shard7_replica_n66", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node69":{ "core":"testNodeLost_shard7_replica_n69", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node68":{ "core":"testNodeLost_shard7_replica_n68", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"cccc0000-d998ffff", "state":"active"}, "shard8":{ "replicas":{ "core_node80":{ "core":"testNodeLost_shard8_replica_n80", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node72":{ "core":"testNodeLost_shard8_replica_n72", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node71":{ "core":"testNodeLost_shard8_replica_n71", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node74":{ "core":"testNodeLost_shard8_replica_n74", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node73":{ "core":"testNodeLost_shard8_replica_n73", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node76":{ "core":"testNodeLost_shard8_replica_n76", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node75":{ "core":"testNodeLost_shard8_replica_n75", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node78":{ "core":"testNodeLost_shard8_replica_n78", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node77":{ "core":"testNodeLost_shard8_replica_n77", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node79":{ "core":"testNodeLost_shard8_replica_n79", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"d9990000-e665ffff", "state":"active"}, "shard9":{ "replicas":{ "core_node90":{ "core":"testNodeLost_shard9_replica_n90", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node81":{ "core":"testNodeLost_shard9_replica_n81", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node83":{ "core":"testNodeLost_shard9_replica_n83", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node82":{ "core":"testNodeLost_shard9_replica_n82", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node85":{ "core":"testNodeLost_shard9_replica_n85", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node84":{ "core":"testNodeLost_shard9_replica_n84", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node87":{ "core":"testNodeLost_shard9_replica_n87", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node86":{ "core":"testNodeLost_shard9_replica_n86", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node89":{ "core":"testNodeLost_shard9_replica_n89", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node88":{ "core":"testNodeLost_shard9_replica_n88", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"e6660000-f332ffff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testNodeLost//clusterstate.json/29)={
"replicationFactor":"10",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"5",
"autoAddReplicas":"false",
"nrtReplicas":"10",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard20":{
"replicas":{
"core_node191":{
"core":"testNodeLost_shard20_replica_n191",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10043_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node200":{
"core":"testNodeLost_shard20_replica_n200",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10033_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node199":{
"core":"testNodeLost_shard20_replica_n199",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10034_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node198":{
"core":"testNodeLost_shard20_replica_n198",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10078_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node197":{
"core":"testNodeLost_shard20_replica_n197",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10094_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node196":{
"core":"testNodeLost_shard20_replica_n196",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10039_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node195":{
"core":"testNodeLost_shard20_replica_n195",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10036_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node194":{
"core":"testNodeLost_shard20_replica_n194",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10015_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node193":{
"core":"testNodeLost_shard20_replica_n193",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10067_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node192":{
"core":"testNodeLost_shard20_replica_n192",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10019_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"73330000-7fffffff",
"state":"active"},
"shard10":{
"replicas":{
"core_node92":{
"core":"testNodeLost_shard10_replica_n92",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10019_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node91":{
"core":"testNodeLost_shard10_replica_n91",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10043_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node94":{
"core":"testNodeLost_shard10_replica_n94",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10015_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node93":{
"core":"testNodeLost_shard10_replica_n93",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10067_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node96":{
"core":"testNodeLost_shard10_replica_n96",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10039_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node95":{
"core":"testNodeLost_shard10_replica_n95",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10036_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node100":{
"core":"testNodeLost_shard10_replica_n100",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10033_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node98":{
"core":"testNodeLost_shard10_replica_n98",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10078_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node97":{
"core":"testNodeLost_shard10_replica_n97",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10094_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node99":{
"core":"testNodeLost_shard10_replica_n99",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10034_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"f3330000-ffffffff",
"state":"active"},
"shard11":{
"replicas":{
"core_node108":{
"core":"testNodeLost_shard11_replica_n108",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10016_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node107":{
"core":"testNodeLost_shard11_replica_n107",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10059_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node106":{
"core":"testNodeLost_shard11_replica_n106",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10097_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node105":{
"core":"testNodeLost_shard11_replica_n105",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10041_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node104":{
"core":"testNodeLost_shard11_replica_n104",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10081_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node103":{
"core":"testNodeLost_shard11_replica_n103",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10058_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node102":{
"core":"testNodeLost_shard11_replica_n102",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10077_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node101":{
"core":"testNodeLost_shard11_replica_n101",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10095_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node110":{
"core":"testNodeLost_shard11_replica_n110",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10066_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node109":{
"core":"testNodeLost_shard11_replica_n109",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10027_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"0-ccbffff",
"state":"active"},
"shard12":{
"replicas":{
"core_node119":{
"core":"testNodeLost_shard12_replica_n119",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10022_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node118":{
"core":"testNodeLost_shard12_replica_n118",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10068_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node117":{
"core":"testNodeLost_shard12_replica_n117",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node116":{
"core":"testNodeLost_shard12_replica_n116",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10046_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node115":{
"core":"testNodeLost_shard12_replica_n115",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10088_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node114":{
"core":"testNodeLost_shard12_replica_n114",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10030_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node113":{
"core":"testNodeLost_shard12_replica_n113",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10085_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node112":{
"core":"testNodeLost_shard12_replica_n112",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10099_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node111":{
"core":"testNodeLost_shard12_replica_n111",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10080_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node120":{
"core":"testNodeLost_shard12_replica_n120",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10032_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"ccc0000-1998ffff",
"state":"active"},
"shard13":{
"replicas":{
"core_node129":{
"core":"testNodeLost_shard13_replica_n129",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10023_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node128":{
"core":"testNodeLost_shard13_replica_n128",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node126":{
"core":"testNodeLost_shard13_replica_n126",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10087_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node125":{
"core":"testNodeLost_shard13_replica_n125",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10060_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node202":{
"node_name":"127.0.0.1:10072_solr",
"core":"testNodeLost_shard13_replica_n201",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node124":{
"core":"testNodeLost_shard13_replica_n124",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10052_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node123":{
"core":"testNodeLost_shard13_replica_n123",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10079_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node122":{
"core":"testNodeLost_shard13_replica_n122",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10062_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node121":{
"core":"testNodeLost_shard13_replica_n121",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10070_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node130":{
"core":"testNodeLost_shard13_replica_n130",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10021_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0}},
"range":"19990000-2665ffff",
"state":"active"},
"shard2":{
"replicas":{
"core_node18":{
"core":"testNodeLost_shard2_replica_n18",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10068_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node17":{
"core":"testNodeLost_shard2_replica_n17",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10004_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node19":{
"core":"testNodeLost_shard2_replica_n19",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10022_solr",
"state":"down",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node20":{
"core":"testNodeLost_shard2_replica_n20",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10032_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node12":{
"core":"testNodeLost_shard2_replica_n12",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10099_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node11":{
"core":"testNodeLost_shard2_replica_n11",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10080_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node14":{
"core":"testNodeLost_shard2_replica_n14",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10030_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node13":{
"core":"testNodeLost_shard2_replica_n13",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10085_solr",
"state":"active",
"type":"NRT",
[...truncated too long message...]
configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null4228494
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null4228494
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null4228494
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1679713142
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1679713142
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1679713142
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 64ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 296 minutes 52 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 64 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/64/
3 tests failed.
FAILED: org.apache.solr.cloud.TestAuthenticationFramework.testBasics
Error Message:
Error from server at http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([E67EF89C67B9ADC0:DBA656B05F57F3B0]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([E67EF89C67B9ADC0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([E67EF89C67B9ADC0]:0)
Build Log:
[...truncated 14290 lines...]
[junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/init-core-data-001
[junit4] 2> 2861823 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001
[junit4] 2> 2861824 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2861825 INFO (Thread-4794) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2861825 INFO (Thread-4794) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2861845 ERROR (Thread-4794) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2861925 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.ZkTestServer start zk server on port:51983
[junit4] 2> 2861958 INFO (zkConnectionManagerCallback-5529-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2861975 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2861993 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 2862080 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2862080 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2862080 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2862086 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ec9e1e6{/solr,null,AVAILABLE}
[junit4] 2> 2862133 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2862133 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2862133 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2862134 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7eb03d29{/solr,null,AVAILABLE}
[junit4] 2> 2862134 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@616caff6{HTTP/1.1,[http/1.1]}{127.0.0.1:53241}
[junit4] 2> 2862134 INFO (jetty-launcher-5526-thread-2) [ ] o.e.j.s.Server Started @2862254ms
[junit4] 2> 2862134 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53241}
[junit4] 2> 2862135 ERROR (jetty-launcher-5526-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2862135 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2862135 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 2862135 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2862135 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2862135 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T19:44:54.740Z
[junit4] 2> 2862143 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2cfdbe7f{HTTP/1.1,[http/1.1]}{127.0.0.1:38720}
[junit4] 2> 2862143 INFO (jetty-launcher-5526-thread-1) [ ] o.e.j.s.Server Started @2862263ms
[junit4] 2> 2862143 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38720}
[junit4] 2> 2862144 ERROR (jetty-launcher-5526-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2862144 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2862144 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 2862144 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2862144 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2862144 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T19:44:54.749Z
[junit4] 2> 2862182 INFO (zkConnectionManagerCallback-5531-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862182 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2862202 INFO (zkConnectionManagerCallback-5533-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862203 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2862227 INFO (jetty-launcher-5526-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51983/solr
[junit4] 2> 2862248 INFO (jetty-launcher-5526-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51983/solr
[junit4] 2> 2862256 INFO (zkConnectionManagerCallback-5537-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862258 INFO (zkConnectionManagerCallback-5541-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862301 INFO (zkConnectionManagerCallback-5543-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862336 INFO (zkConnectionManagerCallback-5545-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862348 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2862351 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38720_solr
[junit4] 2> 2862352 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.Overseer Overseer (id=73839110996361222-127.0.0.1:38720_solr-n_0000000000) starting
[junit4] 2> 2862409 INFO (zkConnectionManagerCallback-5554-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2862412 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51983/solr ready
[junit4] 2> 2862443 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2862443 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38720_solr
[junit4] 2> 2862459 INFO (zkCallback-5544-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2862460 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2862460 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53241_solr
[junit4] 2> 2862479 INFO (zkCallback-5553-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2862494 INFO (zkCallback-5544-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2862507 INFO (zkCallback-5542-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2863017 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
[junit4] 2> 2863173 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38720.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863232 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38720.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863232 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38720.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863237 INFO (jetty-launcher-5526-thread-1) [n:127.0.0.1:38720_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node1/.
[junit4] 2> 2863334 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
[junit4] 2> 2863416 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53241.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863441 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53241.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863442 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53241.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2863463 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node2/.
[junit4] 2> 2863658 INFO (zkConnectionManagerCallback-5559-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2863659 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2863660 INFO (jetty-launcher-5526-thread-2) [n:127.0.0.1:53241_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51983/solr ready
[junit4] 2> 2863718 INFO (zkConnectionManagerCallback-5561-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2863759 INFO (zkConnectionManagerCallback-5566-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2863760 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2863762 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51983/solr ready
[junit4] 2> 2863773 INFO (qtp2107655772-21815) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2863790 INFO (OverseerThreadFactory-7512-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
[junit4] 2> 2863900 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53241/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2863903 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38720/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2863906 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53241/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2863916 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38720/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2864199 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2864200 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2864215 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2864215 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2864238 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2865240 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2865250 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2865266 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2865283 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2865322 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2865328 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
[junit4] 2> 2865333 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema [testcollection_shard2_replica_n6] Schema name=minimal
[junit4] 2> 2865338 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2865341 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2865341 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
[junit4] 2> 2865341 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38720.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2865347 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2865347 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
[junit4] 2> 2865350 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2865350 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n6' using configuration from collection testcollection, trusted=true
[junit4] 2> 2865351 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2865351 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
[junit4] 2> 2865351 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2865351 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node1/testcollection_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node1/./testcollection_shard1_replica_n2/data/]
[junit4] 2> 2865354 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53241.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2865354 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38720.solr.core.testcollection.shard2.replica_n6' (registry 'solr.core.testcollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2865354 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2865355 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [[testcollection_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node1/testcollection_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node1/./testcollection_shard2_replica_n6/data/]
[junit4] 2> 2865357 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53241.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19e39d98
[junit4] 2> 2865357 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2865357 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node2/testcollection_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node2/./testcollection_shard2_replica_n4/data/]
[junit4] 2> 2865359 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2865359 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_E67EF89C67B9ADC0-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
[junit4] 2> 2865707 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2865707 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2865709 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2865709 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2865711 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@18b96960[testcollection_shard1_replica_n2] main]
[junit4] 2> 2865713 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2865713 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2865714 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2865714 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601104142211743744
[junit4] 2> 2865716 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2865716 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2865716 INFO (searcherExecutor-7517-thread-1-processing-n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2 c:testcollection s:shard1 r:core_node5) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@18b96960[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2865717 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2865717 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2865723 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@152b3d8d[testcollection_shard2_replica_n4] main]
[junit4] 2> 2865725 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2865726 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2865727 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2865733 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2865733 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2865735 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2865735 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2865736 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601104142234812416
[junit4] 2> 2865737 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6cad3dc0[testcollection_shard1_replica_n1] main]
[junit4] 2> 2865738 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2865738 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2865739 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2865739 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2865742 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@7408a973[testcollection_shard2_replica_n6] main]
[junit4] 2> 2865743 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2865743 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2865744 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2865745 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601104142244249600
[junit4] 2> 2865747 INFO (searcherExecutor-7520-thread-1-processing-n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1 c:testcollection s:shard1 r:core_node3) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@6cad3dc0[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2865750 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0}, version=0}
[junit4] 2> 2865754 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 2865755 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
[junit4] 2> 2865756 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2865756 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2865759 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2865760 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2865761 INFO (searcherExecutor-7518-thread-1-processing-n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4 c:testcollection s:shard2 r:core_node7) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@152b3d8d[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2865762 INFO (searcherExecutor-7519-thread-1-processing-n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6 c:testcollection s:shard2 r:core_node8) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] Registered new searcher Searcher@7408a973[testcollection_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2865762 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2865763 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601104142263123968
[junit4] 2> 2865771 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
[junit4] 2> 2866261 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2866261 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2866261 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/
[junit4] 2> 2866262 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:38720/solr START replicas=[http://127.0.0.1:53241/solr/testcollection_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2866263 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2866263 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2866264 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53241/solr/testcollection_shard2_replica_n4/
[junit4] 2> 2866264 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n4 url=http://127.0.0.1:53241/solr START replicas=[http://127.0.0.1:38720/solr/testcollection_shard2_replica_n6/] nUpdates=100
[junit4] 2> 2866309 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 2866328 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.S.Request [testcollection_shard2_replica_n6] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=11
[junit4] 2> 2866576 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:38720/solr DONE. We have no versions. sync failed.
[junit4] 2> 2866577 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2866577 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2866577 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2866581 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/ shard1
[junit4] 2> 2866591 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n4 url=http://127.0.0.1:53241/solr DONE. We have no versions. sync failed.
[junit4] 2> 2866591 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2866591 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2866591 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2866657 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53241/solr/testcollection_shard2_replica_n4/ shard2
[junit4] 2> 2866759 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2866762 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2524
[junit4] 2> 2866768 INFO (qtp1366550337-21812) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2552
[junit4] 2> 2866779 INFO (qtp2107655772-21809) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2564
[junit4] 2> 2866790 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2866792 INFO (qtp2107655772-21808) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2592
[junit4] 2> 2866798 INFO (qtp2107655772-21815) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2866896 INFO (zkCallback-5542-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2866897 INFO (zkCallback-5544-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2867793 INFO (OverseerCollectionConfigSetProcessor-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2867798 INFO (qtp2107655772-21815) [n:127.0.0.1:38720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4024
[junit4] 2> 2867798 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 1> replica:core_node3 rstate:active live:true
[junit4] 1> replica:core_node5 rstate:active live:true
[junit4] 1> replica:core_node7 rstate:active live:true
[junit4] 1> replica:core_node8 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 2867799 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testcollection
[junit4] 2> 2867806 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:53241/solr/testcollection_shard2_replica_n4/|http://127.0.0.1:38720/solr/testcollection_shard2_replica_n6/&rows=10&version=2&q=*:*&NOW=1526931900405&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2867807 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.S.Request [testcollection_shard1_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/|http://127.0.0.1:53241/solr/testcollection_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1526931900405&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2867807 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.S.Request [testcollection_shard2_replica_n6] webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:5&wt=javabin&version=2} hits=0 status=0 QTime=7
[junit4] 2> 2867819 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=1, core_node5=1}, version=2}
[junit4] 2> 2867830 INFO (qtp1366550337-21810) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/&wt=javabin&version=2}{add=[1 (1601104144413753344)]} 0 1
[junit4] 2> 2867830 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={_stateVer_=testcollection:5&wt=javabin&version=2}{add=[1 (1601104144413753344)]} 0 16
[junit4] 2> 2867832 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601104144432627712,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2867832 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@155a77c7 commitCommandVersion:1601104144432627712
[junit4] 2> 2867843 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601104144444162048,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2867843 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601104144444162048,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2867843 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@aae9afc commitCommandVersion:1601104144444162048
[junit4] 2> 2867843 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2867846 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2867846 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
[junit4] 2> 2867847 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1601104144448356352,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2867850 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2867851 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b098661[testcollection_shard1_replica_n1] main]
[junit4] 2> 2867852 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2867853 INFO (searcherExecutor-7520-thread-1-processing-n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1 c:testcollection s:shard1 r:core_node3) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@6b098661[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.4.0):C1)))}
[junit4] 2> 2867854 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 21
[junit4] 2> 2867856 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2867859 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n4] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 11
[junit4] 2> 2867860 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d29f481[testcollection_shard1_replica_n2] main]
[junit4] 2> 2867860 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2867862 INFO (searcherExecutor-7517-thread-1-processing-n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2 c:testcollection s:shard1 r:core_node5) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@3d29f481[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.4.0):C1)))}
[junit4] 2> 2867862 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 18
[junit4] 2> 2867862 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2] webapp=/solr path=/update params={_stateVer_=testcollection:5&commit=true&wt=javabin&version=2}{commit=} 0 31
[junit4] 2> 2867865 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:53241/solr/testcollection_shard2_replica_n4/|http://127.0.0.1:38720/solr/testcollection_shard2_replica_n6/&rows=10&version=2&q=*:*&NOW=1526931900468&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 2867879 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.S.Request [testcollection_shard1_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/|http://127.0.0.1:53241/solr/testcollection_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1526931900468&isShard=true&wt=javabin} hits=1 status=0 QTime=0
[junit4] 2> 2867885 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.S.Request [testcollection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=testcollection:5&shards.purpose=64&NOW=1526931900468&ids=1&isShard=true&shard.url=http://127.0.0.1:38720/solr/testcollection_shard1_replica_n2/|http://127.0.0.1:53241/solr/testcollection_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2867885 INFO (qtp1366550337-21824) [n:127.0.0.1:53241_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:5&wt=javabin&version=2} hits=1 status=0 QTime=21
[junit4] 2> 2867886 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=testcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2867893 INFO (OverseerThreadFactory-7512-thread-2) [ ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
[junit4] 2> 2867894 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=null
[junit4] 2> 2867894 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5583a6a: rootName = solr_53241, domain = solr.core.testcollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@6cb1cc1
[junit4] 2> 2867894 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n6, tag=null
[junit4] 2> 2867894 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@388b81e1: rootName = solr_38720, domain = solr.core.testcollection.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.testcollection.shard2.replica_n6 / com.codahale.metrics.MetricRegistry@3d50e8f1
[junit4] 2> 2867932 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n4, tag=null
[junit4] 2> 2867932 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@717d7f0f: rootName = solr_53241, domain = solr.core.testcollection.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.testcollection.shard2.replica_n4 / com.codahale.metrics.MetricRegistry@cbc204b
[junit4] 2> 2867944 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n2, tag=null
[junit4] 2> 2867944 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2b442f02: rootName = solr_38720, domain = solr.core.testcollection.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@5c51f3ca
[junit4] 2> 2867945 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@26183978
[junit4] 2> 2867954 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] CLOSING SolrCore org.apache.solr.core.SolrCore@3b59035c
[junit4] 2> 2867970 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=26183978
[junit4] 2> 2867970 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard1.leader, tag=26183978
[junit4] 2> 2867979 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@4653f830
[junit4] 2> 2867979 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n2, tag=4653f830
[junit4] 2> 2867979 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard1.leader, tag=4653f830
[junit4] 2> 2867979 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n6, tag=3b59035c
[junit4] 2> 2867979 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=3b59035c
[junit4] 2> 2867981 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node5=1}, version=3}
[junit4] 2> 2867982 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
[junit4] 2> 2867983 INFO (qtp1366550337-21823) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=89
[junit4] 2> 2867986 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0}, version=2}
[junit4] 2> 2867987 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
[junit4] 2> 2867987 INFO (qtp2107655772-21903) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard2_replica_n6&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=93
[junit4] 2> 2867988 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] CLOSING SolrCore org.apache.solr.core.SolrCore@575b194
[junit4] 2> 2867988 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n4, tag=575b194
[junit4] 2> 2867988 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=575b194
[junit4] 2> 2868001 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={}, version=4}
[junit4] 2> 2868002 WARN (zkCallback-5542-thread-2) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
[junit4] 2> 2868003 INFO (qtp2107655772-21817) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard1_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=104
[junit4] 2> 2868006 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={}, version=3}
[junit4] 2> 2868007 WARN (zkCallback-5544-thread-1) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
[junit4] 2> 2868017 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard2_replica_n4&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=119
[junit4] 2> 2868120 INFO (zkCallback-5544-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2868121 INFO (zkCallback-5542-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2868729 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=843
[junit4] 2> 2868729 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for collection to disappear - collection: testcollection failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 2> 2868729 INFO (TEST-TestAuthenticationFramework.testBasics-seed#[E67EF89C67B9ADC0]) [ ] o.a.s.c.AbstractDistribZkTestBase Collection has disappeared - collection: testcollection
[junit4] 2> 2868730 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=e97bc853-f476-43d0-8722-249ffd95baa4&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2868732 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=e97bc853-f476-43d0-8722-249ffd95baa4&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2868732 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=e97bc853-f476-43d0-8722-249ffd95baa4&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2868734 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=e97bc853-f476-43d0-8722-249ffd95baa4&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2868750 INFO (OverseerThreadFactory-7512-thread-3) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
[junit4] 2> 2868751 INFO (OverseerCollectionConfigSetProcessor-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2868969 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53241/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2868973 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38720/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2868976 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:53241/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2868991 INFO (OverseerStateUpdate-73839110996361222-127.0.0.1:38720_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n7",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38720/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2869306 INFO (qtp2107655772-21902) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798618683235&coreNodeName=core_node4&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=11
[junit4] 2> 2869306 INFO (parallelCoreAdminExecutor-7513-thread-1-processing-n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n3 e97bc853-f476-43d0-8722-249ffd95baa427183798618683235 CREATE) [n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798618683235&coreNodeName=core_node4&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 2869307 INFO (qtp1366550337-21824) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798615071252&coreNodeName=core_node2&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=11
[junit4] 2> 2869307 INFO (parallelCoreAdminExecutor-7515-thread-2-processing-n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1 e97bc853-f476-43d0-8722-249ffd95baa427183798615071252 CREATE) [n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798615071252&coreNodeName=core_node2&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 2869311 INFO (qtp2107655772-21816) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798636948892&coreNodeName=core_node8&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=16
[junit4] 2> 2869311 INFO (parallelCoreAdminExecutor-7513-thread-2-processing-n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n7 e97bc853-f476-43d0-8722-249ffd95baa427183798636948892 CREATE) [n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798636948892&coreNodeName=core_node8&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 2869327 INFO (qtp1366550337-21820) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798622141627&coreNodeName=core_node6&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=32
[junit4] 2> 2869327 INFO (parallelCoreAdminExecutor-7515-thread-1-processing-n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n5 e97bc853-f476-43d0-8722-249ffd95baa427183798622141627 CREATE) [n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=e97bc853-f476-43d0-8722-249ffd95baa427183798622141627&coreNodeName=core_node6&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 2869328 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : e97bc853-f476-43d0-8722-249ffd95baa427183798622141627
[junit4] 2> 2869328 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=e97bc853-f476-43d0-8722-249ffd95baa427183798622141627&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2869735 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=e97bc853-f476-43d0-8722-249ffd95baa4&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2869736 INFO (qtp2107655772-21814) [n:127.0.0.1:38720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=e97bc853-f476-43d0-8722-249ffd95baa4&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2870321 INFO (parallelCoreAdminExecutor-7513-thread-1-processing-n:127.0.0.1:38720_solr x:testcollection_shard1_replica_n3 e97bc853-f476-43d0-8722-249ffd95baa427183798618683235 CREATE) [n:127.0.0.1:38720_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2870329 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : e97bc853-f476-43d0-8722-249ffd95baa427183798622141627
[junit4] 2> 2870330 INFO (qtp1366550337-21822) [n:127.0.0.1:53241_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=e97bc853-f476-43d0-8722-249ffd95baa427183798622141627&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2870342 INFO (parallelCoreAdminExecutor-7513-thread-2-processing-n:127.0.0.1:38720_solr x:testcollection_shard2_replica_n7 e97bc853-f476-43d0-8722-249ffd95baa427183798636948892 CREATE) [n:127.0.0.1:38720_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2870388 INFO (parallelCoreAdminExecutor-7515-thread-1-processing-n:127.0.0.1:53241_solr x:testcollection_shard2_replica_n5 e97bc853-f476-43d0-8722-249ffd95baa427183798622141627 CREATE) [n:127.0.0.1:53241_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2870393 INFO (parallelCoreAdminExecutor-7515-thread-2-processing-n:127.0.0.1:53241_solr x:testcollection_shard1_replica_n1 e97bc853-f476-43d0-8722-249ffd95baa427183
[...truncated too long message...]
owed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null323693840
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null323693840
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null323693840
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1948597150
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1948597150
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1948597150
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 62ms :: artifacts dl 6ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 270 minutes 52 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 63 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/63/
4 tests failed.
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection
Error Message:
Error from server at http://127.0.0.1:43213/solr: KeeperErrorCode = Session expired for /configs/conf
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43213/solr: KeeperErrorCode = Session expired for /configs/conf
at __randomizedtesting.SeedInfo.seed([63B30E99B809974F:4933627C0FD3B787]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi
Error Message:
expected:<127.0.0.1:[35843]_solr> but was:<127.0.0.1:[42067]_solr>
Stack Trace:
org.junit.ComparisonFailure: expected:<127.0.0.1:[35843]_solr> but was:<127.0.0.1:[42067]_solr>
at __randomizedtesting.SeedInfo.seed([63B30E99B809974F:349AF52C63FB7554]:0)
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi(AutoScalingHandlerTest.java:851)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
Error Message:
should be at least one inactive event
Stack Trace:
java.lang.AssertionError: should be at least one inactive event
at __randomizedtesting.SeedInfo.seed([63B30E99B809974F:7E9FCEEBD94AB044]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([63B30E99B809974F:78381B21C6E462]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger(SearchRateTriggerTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12887 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_63B30E99B809974F-001/init-core-data-001
[junit4] 2> 1528971 WARN (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=43 numCloses=43
[junit4] 2> 1528971 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1528973 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1528973 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1528974 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_63B30E99B809974F-001/tempDir-001
[junit4] 2> 1528974 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1529068 INFO (Thread-1285) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1529068 INFO (Thread-1285) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1529132 ERROR (Thread-1285) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1529205 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37659
[junit4] 2> 1529234 INFO (zkConnectionManagerCallback-1277-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1529239 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1529240 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1529240 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1529240 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1529240 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cefb319{/solr,null,AVAILABLE}
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@56c4954{SSL,[ssl, http/1.1]}{127.0.0.1:39021}
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.e.j.s.Server Started @1529431ms
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39021}
[junit4] 2> 1529241 ERROR (jetty-launcher-1274-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1529241 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T21:03:25.156Z
[junit4] 2> 1529272 INFO (zkConnectionManagerCallback-1279-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1529273 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1529297 INFO (jetty-launcher-1274-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37659/solr
[junit4] 2> 1529311 INFO (zkConnectionManagerCallback-1283-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1529313 INFO (zkConnectionManagerCallback-1285-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1529566 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1529576 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39021_solr
[junit4] 2> 1529577 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.Overseer Overseer (id=72659712980811779-127.0.0.1:39021_solr-n_0000000000) starting
[junit4] 2> 1529604 INFO (zkConnectionManagerCallback-1292-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1529606 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37659/solr ready
[junit4] 2> 1529606 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39021_solr
[junit4] 2> 1529615 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1529688 INFO (zkCallback-1284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1529688 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1529689 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1529689 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1529724 INFO (zkCallback-1291-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1529978 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:39021_solr]
[junit4] 2> 1529996 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1530013 DEBUG (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1530013 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1530375 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39021.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19129136
[junit4] 2> 1530412 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39021.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19129136
[junit4] 2> 1530412 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39021.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19129136
[junit4] 2> 1530413 INFO (jetty-launcher-1274-thread-1) [n:127.0.0.1:39021_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_63B30E99B809974F-001/tempDir-001/node1/.
[junit4] 2> 1530581 INFO (zkConnectionManagerCallback-1294-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1530632 INFO (zkConnectionManagerCallback-1299-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1530633 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1530652 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37659/solr ready
[junit4] 2> 1530802 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1530802 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1530803 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1530803 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1530836 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 1530848 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1530849 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1530849 DEBUG (ScheduledTrigger-1841-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1530878 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1530914 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1530950 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1530987 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531025 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531029 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531073 INFO (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[63B30E99B809974F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testInactiveShardCleanup
[junit4] 2> 1531074 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531095 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531115 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531145 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531166 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531202 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531247 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531275 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531311 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531348 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531385 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531421 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531441 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531475 DEBUG (simCloudManagerPool-1840-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1: {"core_node1":{
[junit4] 2> "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
[junit4] 2> 1531475 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531475 DEBUG (simCloudManagerPool-1840-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_1: {"core_node5":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_1_replica0",
[junit4] 2> "shard":"shard1_1",
[junit4] 2> "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
[junit4] 2> 1531475 DEBUG (simCloudManagerPool-1840-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_0: {"core_node4":{
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_0_replica0",
[junit4] 2> "shard":"shard1_0",
[junit4] 2> "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
[junit4] 2> 1531510 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531514 DEBUG (simCloudManagerPool-1840-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1531532 DEBUG (simCloudManagerPool-1840-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1531568 DEBUG (ScheduledTrigger-1841-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531604 DEBUG (ScheduledTrigger-1841-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1531604 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1531604 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1531604 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1531621 DEBUG (simCloudManagerPool-1840-thread-9) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1531621 DEBUG (simCloudManagerPool-1840-thread-9) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1531622 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1531623 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1531623 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1531623 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger .scheduled_maintenance firing registered processor for scheduled time 2018-05-20T21:03:34.284Z, now=2018-05-20T21:04:13.305Z
[junit4] 2> 1531623 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{"actualEventTime":1526850253305}}
[junit4] 2> 1531657 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance]
[junit4] 2> 1531661 INFO (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=9187656162686429, stage=STARTED, actionName='null', event={
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}}, context={}, config={
[junit4] 2> "trigger":".scheduled_maintenance",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":"inactive_shard_plan",
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":"inactive_shard_plan"}, message='null'}
[junit4] 2> 1531677 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}}
[junit4] 2> 1531693 INFO (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=9187657747784079, stage=BEFORE_ACTION, actionName='inactive_shard_plan', event={
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}}, context={properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance}, config={
[junit4] 2> "trigger":".scheduled_maintenance",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":"inactive_shard_plan",
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":"inactive_shard_plan"}, message='null'}
[junit4] 2> 1531695 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.InactiveShardPlanAction ScheduledMaintenanceTriggerTest_collection1/shard1: tstamp=1526850240409407850, time=1526850256904598950, delta=16
[junit4] 2> 1531695 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.InactiveShardPlanAction -- delete inactive ScheduledMaintenanceTriggerTest_collection1 / shard1
[junit4] 2> 1531730 INFO (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=9187659608673629, stage=AFTER_ACTION, actionName='inactive_shard_plan', event={
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard, method=GET, params.action=DELETESHARD, params.collection=ScheduledMaintenanceTriggerTest_collection1, params.shard=shard1}], properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance, properties.inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, properties.AFTER_ACTION=[inactive_shard_plan]}, config={
[junit4] 2> "trigger":".scheduled_maintenance",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":"inactive_shard_plan",
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":"inactive_shard_plan"}, message='null'}
[junit4] 2> 1531730 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard@5896b07b], inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, AFTER_ACTION=[inactive_shard_plan], BEFORE_ACTION=[inactive_shard_plan, execute_plan]}
[junit4] 2> 1531731 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=DELETESHARD&collection=ScheduledMaintenanceTriggerTest_collection1&shard=shard1
[junit4] 2> 1531929 INFO (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=9187669527683779, stage=SUCCEEDED, actionName='null', event={
[junit4] 2> "id":"20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9",
[junit4] 2> "source":".scheduled_maintenance",
[junit4] 2> "eventTime":9187654263847279,
[junit4] 2> "eventType":"SCHEDULED",
[junit4] 2> "properties":{
[junit4] 2> "actualEventTime":1526850253305,
[junit4] 2> "_enqueue_time_":9187655950777879}}, context={}, config={
[junit4] 2> "trigger":".scheduled_maintenance",
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED"],
[junit4] 2> "afterAction":"inactive_shard_plan",
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
[junit4] 2> "beforeAction":"inactive_shard_plan"}, message='null'}
[junit4] 2> 1531929 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1531929 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
[junit4] 2> 1531929 DEBUG (AutoscalingActionExecutor-1842-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 12596 ms for event id=20a41f74b7396fTa79rrmjuengi0w5nbt2caiep9
[junit4] 2> 1531950 DEBUG (simCloudManagerPool-1840-thread-15) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1531950 DEBUG (simCloudManagerPool-1840-thread-15) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1531951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1531965 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1531965 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1531986 DEBUG (simCloudManagerPool-1840-thread-16) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1531986 DEBUG (simCloudManagerPool-1840-thread-16) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1531987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1531987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1531987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1531987 INFO (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[63B30E99B809974F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testInactiveShardCleanup
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=63B30E99B809974F -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=it -Dtests.timezone=Asia/Hovd -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.99s J1 | ScheduledMaintenanceTriggerTest.testInactiveShardCleanup <<<
[junit4] > Throwable #1: java.lang.AssertionError: should be at least one inactive event
[junit4] > at __randomizedtesting.SeedInfo.seed([63B30E99B809974F:7E9FCEEBD94AB044]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1532058 DEBUG (ScheduledTrigger-1841-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532078 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532096 DEBUG (ScheduledTrigger-1841-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532131 DEBUG (ScheduledTrigger-1841-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532168 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532237 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532257 DEBUG (ScheduledTrigger-1841-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532294 DEBUG (ScheduledTrigger-1841-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1532297 INFO (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[63B30E99B809974F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerDefaults
[junit4] 2> 1532297 INFO (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[63B30E99B809974F]) [ ] o.a.s.c.a.ScheduledMaintenanceTriggerTest {
[junit4] 2> "cluster-preferences":[
[junit4] 2> {
[junit4] 2> "minimize":"cores",
[junit4] 2> "precision":1},
[junit4] 2> {"maximize":"freedisk"}],
[junit4] 2> "triggers":{
[junit4] 2> ".scheduled_maintenance":{
[junit4] 2> "startTime":"NOW",
[junit4] 2> "event":"scheduled",
[junit4] 2> "every":"+1DAY",
[junit4] 2> "actions":[
[junit4] 2> {
[junit4] 2> "name":"inactive_shard_plan",
[junit4] 2> "class":"solr.InactiveShardPlanAction"},
[junit4] 2> {
[junit4] 2> "name":"execute_plan",
[junit4] 2> "class":"solr.ExecutePlanAction"}],
[junit4] 2> "enabled":true},
[junit4] 2> ".auto_add_replicas":{
[junit4] 2> "name":".auto_add_replicas",
[junit4] 2> "event":"nodeLost",
[junit4] 2> "waitFor":120,
[junit4] 2> "actions":[
[junit4] 2> {
[junit4] 2> "name":"auto_add_replicas_plan",
[junit4] 2> "class":"solr.AutoAddReplicasPlanAction"},
[junit4] 2> {
[junit4] 2> "name":"execute_plan",
[junit4] 2> "class":"solr.ExecutePlanAction"}],
[junit4] 2> "enabled":true}},
[junit4] 2> "listeners":{
[junit4] 2> ".scheduled_maintenance.system":{
[junit4] 2> "trigger":".scheduled_maintenance",
[junit4] 2> "afterAction":[],
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED",
[junit4] 2> "BEFORE_ACTION",
[junit4] 2> "AFTER_ACTION",
[junit4] 2> "IGNORED"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] 2> "beforeAction":[]},
[junit4] 2> ".auto_add_replicas.system":{
[junit4] 2> "trigger":".auto_add_replicas",
[junit4] 2> "afterAction":[],
[junit4] 2> "stage":[
[junit4] 2> "STARTED",
[junit4] 2> "ABORTED",
[junit4] 2> "SUCCEEDED",
[junit4] 2> "FAILED",
[junit4] 2> "BEFORE_ACTION",
[junit4] 2> "AFTER_ACTION",
[junit4] 2> "IGNORED"],
[junit4] 2> "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] 2> "beforeAction":[]}},
[junit4] 2> "properties":{}}
[junit4] 2> 1532315 INFO (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[63B30E99B809974F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerDefaults
[junit4] 2> 1532315 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1532315 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1532315 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1532316 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1532316 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1532316 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1532316 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1532424 INFO (jetty-closer-1275-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@56c4954{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1532425 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1650301011
[junit4] 2> 1532440 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1532440 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5bc38f3: rootName = solr_39021, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7d4d6261
[junit4] 2> 1532461 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1532462 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@540cbe65: rootName = solr_39021, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2d8430fa
[junit4] 2> 1532477 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1532477 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fe289ef: rootName = solr_39021, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@8de3737
[junit4] 2> 1532479 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39021_solr
[junit4] 2> 1532480 INFO (zkCallback-1291-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1532480 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1532480 INFO (zkCallback-1284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1532493 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1532493 INFO (jetty-closer-1275-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72659712980811779-127.0.0.1:39021_solr-n_0000000000) closing
[junit4] 2> 1532493 INFO (OverseerStateUpdate-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39021_solr
[junit4] 2> 1532497 WARN (OverseerAutoScalingTriggerThread-72659712980811779-127.0.0.1:39021_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1532498 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1532498 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1532498 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1532498 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1532517 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1532517 DEBUG (jetty-closer-1275-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1532550 INFO (jetty-closer-1275-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5cefb319{/solr,null,UNAVAILABLE}
[junit4] 2> 1532550 INFO (jetty-closer-1275-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1532594 ERROR (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1532600 INFO (SUITE-ScheduledMaintenanceTriggerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37659 37659
[junit4] 2> 1532842 INFO (Thread-1285) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37659 37659
[junit4] 2> 1532843 WARN (Thread-1285) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_63B30E99B809974F-001
[junit4] 2> May 20, 2018 9:03:28 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1347, maxMBSortInHeap=6.8741488413102525, sim=RandomSimilarity(queryNorm=true): {}, locale=it, timezone=Asia/Hovd
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=117805000,total=466616320
[junit4] 2> NOTE: All tests run in this JVM: [SimplePostToolTest, BasicAuthIntegrationTest, TestCloudInspectUtil, BasicFunctionalityTest, LIROnShardRestartTest, TestEmbeddedSolrServerSchemaAPI, SliceStateTest, TestFuzzyAnalyzedSuggestions, SSLMigrationTest, SolrMetricManagerTest, TestZkChroot, LeaderFailoverAfterPartitionTest, TestPointFields, SolrCLIZkUtilsTest, GraphQueryTest, SpatialHeatmapFacetsTest, TestStressRecovery, TestSha256AuthenticationProvider, AutoAddReplicasPlanActionTest, TestRandomCollapseQParserPlugin, DeleteReplicaTest, SolrShardReporterTest, SystemInfoHandlerTest, ConcurrentDeleteAndCreateCollectionTest, SortByFunctionTest, ChaosMonkeySafeLeaderTest, TestReversedWildcardFilterFactory, TestCloudPseudoReturnFields, ZkShardTermsTest, TestRTGBase, CoreAdminCreateDiscoverTest, AtomicUpdatesTest, TestManagedSynonymGraphFilterFactory, HdfsRestartWhileUpdatingTest, BadComponentTest, ChangedSchemaMergeTest, DocValuesMissingTest, TestTrie, SearchRateTriggerIntegrationTest, ScheduledMaintenanceTriggerTest]
[junit4] Completed [154/805 (1!)] on J1 in 4.51s, 2 tests, 1 failure <<< FAILURES!
[...truncated 272 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_63B30E99B809974F-001/init-core-data-001
[junit4] 2> 1899355 WARN (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=42 numCloses=42
[junit4] 2> 1899355 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1899356 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1899379 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1899379 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_63B30E99B809974F-001/tempDir-001
[junit4] 2> 1899379 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1899406 INFO (Thread-2275) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1899406 INFO (Thread-2275) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1899503 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42701
[junit4] 2> 1899504 ERROR (Thread-2275) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1899938 INFO (zkConnectionManagerCallback-1974-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1899970 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1900005 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 1900093 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1900093 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1900093 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1900094 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36c99ce6{/solr,null,AVAILABLE}
[junit4] 2> 1900094 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6406c0fc{SSL,[ssl, http/1.1]}{127.0.0.1:45765}
[junit4] 2> 1900094 INFO (jetty-launcher-1971-thread-2) [ ] o.e.j.s.Server Started @1900257ms
[junit4] 2> 1900094 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45765}
[junit4] 2> 1900095 ERROR (jetty-launcher-1971-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1900095 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1900095 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1900095 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1900095 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1900095 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T21:09:35.998Z
[junit4] 2> 1900182 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1900182 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1900182 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1900183 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6784033{/solr,null,AVAILABLE}
[junit4] 2> 1900183 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79b98838{SSL,[ssl, http/1.1]}{127.0.0.1:42067}
[junit4] 2> 1900183 INFO (jetty-launcher-1971-thread-1) [ ] o.e.j.s.Server Started @1900346ms
[junit4] 2> 1900183 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42067}
[junit4] 2> 1900184 ERROR (jetty-launcher-1971-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1900184 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1900184 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 1900184 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1900184 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1900184 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T21:09:36.087Z
[junit4] 2> 1900385 INFO (zkConnectionManagerCallback-1976-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900386 INFO (zkConnectionManagerCallback-1978-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900390 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1900402 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1900597 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10223a5646f0001, likely client has closed socket
[junit4] 2> 1900633 INFO (jetty-launcher-1971-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42701/solr
[junit4] 2> 1900650 INFO (jetty-launcher-1971-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42701/solr
[junit4] 2> 1900652 INFO (zkConnectionManagerCallback-1982-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900668 INFO (zkConnectionManagerCallback-1986-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900688 INFO (zkConnectionManagerCallback-1988-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900745 INFO (zkConnectionManagerCallback-1990-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1900977 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1900978 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42067_solr
[junit4] 2> 1900992 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.Overseer Overseer (id=72659737252921349-127.0.0.1:42067_solr-n_0000000000) starting
[junit4] 2> 1901069 INFO (zkConnectionManagerCallback-1999-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1901071 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42701/solr ready
[junit4] 2> 1901088 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42067_solr
[junit4] 2> 1901153 INFO (OverseerStateUpdate-72659737252921349-127.0.0.1:42067_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1901227 INFO (zkCallback-1998-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1901401 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1901455 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1901602 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1901602 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45765_solr
[junit4] 2> 1901603 INFO (zkCallback-1998-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1901836 INFO (zkCallback-1987-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1901836 INFO (zkCallback-1989-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1903076 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45765.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903136 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45765.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903136 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45765.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903170 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_63B30E99B809974F-001/tempDir-001/node2/.
[junit4] 2> 1903244 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42067.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903267 INFO (zkConnectionManagerCallback-2004-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1903268 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1903269 INFO (jetty-launcher-1971-thread-2) [n:127.0.0.1:45765_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42701/solr ready
[junit4] 2> 1903283 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42067.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903283 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42067.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45791e5f
[junit4] 2> 1903284 INFO (jetty-launcher-1971-thread-1) [n:127.0.0.1:42067_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_63B30E99B809974F-001/tempDir-001/node1/.
[junit4] 2> 1903513 INFO (zkConnectionManagerCallback-2006-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1903529 INFO (zkConnectionManagerCallback-2011-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1903530 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1903543 INFO (SUITE-AutoScalingHandlerTest-seed#[63B30E99B809974F]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42701/solr ready
[junit4] 2> 1903719 INFO (TEST-AutoScalingHandlerTest.testConcurrentUpdates-seed#[63B30E99B809974F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testConcurrentUpdates
[junit4] 2> 1904247 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=16
[junit4] 2> 1904503 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904524 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904566 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904579 INFO (qtp571303410-8283) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=457
[junit4] 2> 1904626 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904694 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904709 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904711 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904718 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=137
[junit4] 2> 1904720 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904730 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904754 INFO (qtp571303410-8282) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=359
[junit4] 2> 1904755 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904767 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904772 INFO (qtp571303410-8282) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904858 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=83
[junit4] 2> 1904861 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=90
[junit4] 2> 1904876 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=14
[junit4] 2> 1904877 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904913 INFO (qtp571303410-8285) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904945 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904968 INFO (qtp571303410-8282) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=92
[junit4] 2> 1904984 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904985 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1904987 INFO (qtp571303410-8283) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=260
[junit4] 2> 1905009 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905112 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905113 INFO (qtp571303410-8284) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905128 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=121
[junit4] 2> 1905130 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905240 INFO (qtp571303410-8284) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905240 INFO (qtp571303410-8283) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905241 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=112
[junit4] 2> 1905257 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905287 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=29
[junit4] 2> 1905364 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=338
[junit4] 2> 1905367 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905421 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905421 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905433 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905435 INFO (qtp790244301-8294) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=70
[junit4] 2> 1905477 INFO (qtp571303410-8283) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905478 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905577 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905578 INFO (qtp571303410-8282) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=209
[junit4] 2> 1905579 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905587 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905588 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905616 INFO (qtp571303410-8284) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=148
[junit4] 2> 1905945 INFO (qtp790244301-8293) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905946 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905952 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=92
[junit4] 2> 1905971 INFO (qtp571303410-8281) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905972 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905973 INFO (qtp790244301-8296) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1905990 INFO (qtp571303410-8283) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 1906026 INFO (qtp571303410-8354) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=53
[junit4] 2> 1906127 INFO (qtp790244301-8289) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1906131 INFO (qtp790244301-8291) [n:127.0.0.1:42067_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1906178 INFO (qtp571303410-8282) [n:127.0.0.1:45765_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1906178 INFO (qtp790244301-8296)
[...truncated too long message...]
ure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null266837235
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null266837235
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null266837235
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1572200061
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1572200061
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1572200061
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 52ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 293 minutes 45 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 62 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/62/
7 tests failed.
FAILED: org.apache.solr.cloud.TestAuthenticationFramework.testBasics
Error Message:
Error from server at http://127.0.0.1:41715/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:41715/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([3954E34127A9E6C4:48C4D6D1F47B8B4]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41715/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple2 null Live Nodes: [127.0.0.1:41379_solr, 127.0.0.1:44234_solr] Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/23)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node2":{ "dataDir":"hdfs://localhost:57014/data/testSimple2/core_node2/data/", "base_url":"http://127.0.0.1:44234/solr", "node_name":"127.0.0.1:44234_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node2/data/tlog", "core":"testSimple2_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node4":{ "dataDir":"hdfs://localhost:57014/data/testSimple2/core_node4/data/", "base_url":"http://127.0.0.1:52293/solr", "node_name":"127.0.0.1:52293_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node4/data/tlog", "core":"testSimple2_shard1_replica_n3", "shared_storage":"true", "state":"down"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node6":{ "dataDir":"hdfs://localhost:57014/data/testSimple2/core_node6/data/", "base_url":"http://127.0.0.1:44234/solr", "node_name":"127.0.0.1:44234_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node6/data/tlog", "core":"testSimple2_shard2_replica_n5", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://localhost:57014/data/testSimple2/core_node8/data/", "base_url":"http://127.0.0.1:52293/solr", "node_name":"127.0.0.1:52293_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node8/data/tlog", "core":"testSimple2_shard2_replica_n7", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple2
null
Live Nodes: [127.0.0.1:41379_solr, 127.0.0.1:44234_solr]
Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/23)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node2":{
"dataDir":"hdfs://localhost:57014/data/testSimple2/core_node2/data/",
"base_url":"http://127.0.0.1:44234/solr",
"node_name":"127.0.0.1:44234_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node2/data/tlog",
"core":"testSimple2_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node4":{
"dataDir":"hdfs://localhost:57014/data/testSimple2/core_node4/data/",
"base_url":"http://127.0.0.1:52293/solr",
"node_name":"127.0.0.1:52293_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node4/data/tlog",
"core":"testSimple2_shard1_replica_n3",
"shared_storage":"true",
"state":"down"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node6":{
"dataDir":"hdfs://localhost:57014/data/testSimple2/core_node6/data/",
"base_url":"http://127.0.0.1:44234/solr",
"node_name":"127.0.0.1:44234_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node6/data/tlog",
"core":"testSimple2_shard2_replica_n5",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://localhost:57014/data/testSimple2/core_node8/data/",
"base_url":"http://127.0.0.1:52293/solr",
"node_name":"127.0.0.1:52293_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:57014/data/testSimple2/core_node8/data/tlog",
"core":"testSimple2_shard2_replica_n7",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([3954E34127A9E6C4:1E7C7BF005A3215]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([3954E34127A9E6C4]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([3954E34127A9E6C4]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
there should be new MOVERPLICA ops
Stack Trace:
java.lang.AssertionError: there should be new MOVERPLICA ops
at __randomizedtesting.SeedInfo.seed([3954E34127A9E6C4:92AEFE54F87560EA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
last state: DocCollection(testNodeLost//clusterstate.json/93)={ "replicationFactor":"10", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"5", "autoAddReplicas":"false", "nrtReplicas":"10", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard20":{ "replicas":{ "core_node191":{ "core":"testNodeLost_shard20_replica_n191", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node258":{ "node_name":"127.0.0.1:10009_solr", "core":"testNodeLost_shard20_replica_n257", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node200":{ "core":"testNodeLost_shard20_replica_n200", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node266":{ "node_name":"127.0.0.1:10028_solr", "core":"testNodeLost_shard20_replica_n265", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node199":{ "core":"testNodeLost_shard20_replica_n199", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node196":{ "core":"testNodeLost_shard20_replica_n196", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node195":{ "core":"testNodeLost_shard20_replica_n195", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node250":{ "node_name":"127.0.0.1:10011_solr", "core":"testNodeLost_shard20_replica_n249", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node193":{ "core":"testNodeLost_shard20_replica_n193", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node192":{ "core":"testNodeLost_shard20_replica_n192", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"73330000-7fffffff", "state":"active"}, "shard10":{ "replicas":{ "core_node92":{ "core":"testNodeLost_shard10_replica_n92", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node91":{ "core":"testNodeLost_shard10_replica_n91", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node93":{ "core":"testNodeLost_shard10_replica_n93", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node268":{ "node_name":"127.0.0.1:10044_solr", "core":"testNodeLost_shard10_replica_n267", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node96":{ "core":"testNodeLost_shard10_replica_n96", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10071_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node95":{ "core":"testNodeLost_shard10_replica_n95", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node100":{ "core":"testNodeLost_shard10_replica_n100", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10028_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node252":{ "node_name":"127.0.0.1:10014_solr", "core":"testNodeLost_shard10_replica_n251", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node99":{ "core":"testNodeLost_shard10_replica_n99", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10074_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node260":{ "node_name":"127.0.0.1:10087_solr", "core":"testNodeLost_shard10_replica_n259", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"f3330000-ffffffff", "state":"active"}, "shard11":{ "replicas":{ "core_node108":{ "core":"testNodeLost_shard11_replica_n108", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node107":{ "core":"testNodeLost_shard11_replica_n107", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node105":{ "core":"testNodeLost_shard11_replica_n105", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node104":{ "core":"testNodeLost_shard11_replica_n104", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node236":{ "node_name":"127.0.0.1:10069_solr", "core":"testNodeLost_shard11_replica_n235", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node103":{ "core":"testNodeLost_shard11_replica_n103", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node102":{ "core":"testNodeLost_shard11_replica_n102", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node101":{ "core":"testNodeLost_shard11_replica_n101", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node110":{ "core":"testNodeLost_shard11_replica_n110", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node109":{ "core":"testNodeLost_shard11_replica_n109", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"0-ccbffff", "state":"active"}, "shard12":{ "replicas":{ "core_node119":{ "core":"testNodeLost_shard12_replica_n119", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node118":{ "core":"testNodeLost_shard12_replica_n118", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node216":{ "node_name":"127.0.0.1:10034_solr", "core":"testNodeLost_shard12_replica_n215", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node115":{ "core":"testNodeLost_shard12_replica_n115", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node114":{ "core":"testNodeLost_shard12_replica_n114", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node113":{ "core":"testNodeLost_shard12_replica_n113", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node212":{ "node_name":"127.0.0.1:10094_solr", "core":"testNodeLost_shard12_replica_n211", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node112":{ "core":"testNodeLost_shard12_replica_n112", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node120":{ "core":"testNodeLost_shard12_replica_n120", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node208":{ "node_name":"127.0.0.1:10053_solr", "core":"testNodeLost_shard12_replica_n207", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"ccc0000-1998ffff", "state":"active"}, "shard2":{ "replicas":{ "core_node18":{ "core":"testNodeLost_shard2_replica_n18", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10083_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node206":{ "node_name":"127.0.0.1:10059_solr", "core":"testNodeLost_shard2_replica_n205", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node19":{ "core":"testNodeLost_shard2_replica_n19", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10032_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node214":{ "node_name":"127.0.0.1:10091_solr", "core":"testNodeLost_shard2_replica_n213", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node210":{ "node_name":"127.0.0.1:10037_solr", "core":"testNodeLost_shard2_replica_n209", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node20":{ "core":"testNodeLost_shard2_replica_n20", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node12":{ "core":"testNodeLost_shard2_replica_n12", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node14":{ "core":"testNodeLost_shard2_replica_n14", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node13":{ "core":"testNodeLost_shard2_replica_n13", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node15":{ "core":"testNodeLost_shard2_replica_n15", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"8ccc0000-9998ffff", "state":"active"}, "shard13":{ "replicas":{ "core_node129":{ "core":"testNodeLost_shard13_replica_n129", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node128":{ "core":"testNodeLost_shard13_replica_n128", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node127":{ "core":"testNodeLost_shard13_replica_n127", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node126":{ "core":"testNodeLost_shard13_replica_n126", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node125":{ "core":"testNodeLost_shard13_replica_n125", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node124":{ "core":"testNodeLost_shard13_replica_n124", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node123":{ "core":"testNodeLost_shard13_replica_n123", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node122":{ "core":"testNodeLost_shard13_replica_n122", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node121":{ "core":"testNodeLost_shard13_replica_n121", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node130":{ "core":"testNodeLost_shard13_replica_n130", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"19990000-2665ffff", "state":"active"}, "shard14":{ "replicas":{ "core_node139":{ "core":"testNodeLost_shard14_replica_n139", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node138":{ "core":"testNodeLost_shard14_replica_n138", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node137":{ "core":"testNodeLost_shard14_replica_n137", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node135":{ "core":"testNodeLost_shard14_replica_n135", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node134":{ "core":"testNodeLost_shard14_replica_n134", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node133":{ "core":"testNodeLost_shard14_replica_n133", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node132":{ "core":"testNodeLost_shard14_replica_n132", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node131":{ "core":"testNodeLost_shard14_replica_n131", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node262":{ "node_name":"127.0.0.1:10041_solr", "core":"testNodeLost_shard14_replica_n261", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node140":{ "core":"testNodeLost_shard14_replica_n140", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"26660000-3332ffff", "state":"active"}, "shard3":{ "replicas":{ "core_node29":{ "core":"testNodeLost_shard3_replica_n29", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node28":{ "core":"testNodeLost_shard3_replica_n28", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node30":{ "core":"testNodeLost_shard3_replica_n30", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node21":{ "core":"testNodeLost_shard3_replica_n21", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node23":{ "core":"testNodeLost_shard3_replica_n23", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node22":{ "core":"testNodeLost_shard3_replica_n22", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node25":{ "core":"testNodeLost_shard3_replica_n25", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node24":{ "core":"testNodeLost_shard3_replica_n24", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10069_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node27":{ "core":"testNodeLost_shard3_replica_n27", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node26":{ "core":"testNodeLost_shard3_replica_n26", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10009_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"99990000-a665ffff", "state":"active"}, "shard15":{ "replicas":{ "core_node149":{ "core":"testNodeLost_shard15_replica_n149", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node147":{ "core":"testNodeLost_shard15_replica_n147", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node246":{ "node_name":"127.0.0.1:10085_solr", "core":"testNodeLost_shard15_replica_n245", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node146":{ "core":"testNodeLost_shard15_replica_n146", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node145":{ "core":"testNodeLost_shard15_replica_n145", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node144":{ "core":"testNodeLost_shard15_replica_n144", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node143":{ "core":"testNodeLost_shard15_replica_n143", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node142":{ "core":"testNodeLost_shard15_replica_n142", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node141":{ "core":"testNodeLost_shard15_replica_n141", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node150":{ "core":"testNodeLost_shard15_replica_n150", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"33330000-3fffffff", "state":"active"}, "shard4":{ "replicas":{ "core_node39":{ "core":"testNodeLost_shard4_replica_n39", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node40":{ "core":"testNodeLost_shard4_replica_n40", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node32":{ "core":"testNodeLost_shard4_replica_n32", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node31":{ "core":"testNodeLost_shard4_replica_n31", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node264":{ "node_name":"127.0.0.1:10092_solr", "core":"testNodeLost_shard4_replica_n263", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node34":{ "core":"testNodeLost_shard4_replica_n34", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node33":{ "core":"testNodeLost_shard4_replica_n33", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node35":{ "core":"testNodeLost_shard4_replica_n35", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node38":{ "core":"testNodeLost_shard4_replica_n38", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10063_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node37":{ "core":"testNodeLost_shard4_replica_n37", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"a6660000-b332ffff", "state":"active"}, "shard16":{ "replicas":{ "core_node218":{ "node_name":"127.0.0.1:10031_solr", "core":"testNodeLost_shard16_replica_n217", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node238":{ "node_name":"127.0.0.1:10015_solr", "core":"testNodeLost_shard16_replica_n237", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node159":{ "core":"testNodeLost_shard16_replica_n159", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node202":{ "node_name":"127.0.0.1:10072_solr", "core":"testNodeLost_shard16_replica_n201", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node156":{ "core":"testNodeLost_shard16_replica_n156", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node155":{ "core":"testNodeLost_shard16_replica_n155", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node242":{ "node_name":"127.0.0.1:10068_solr", "core":"testNodeLost_shard16_replica_n241", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node153":{ "core":"testNodeLost_shard16_replica_n153", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node151":{ "core":"testNodeLost_shard16_replica_n151", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node160":{ "core":"testNodeLost_shard16_replica_n160", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"40000000-4ccbffff", "state":"active"}, "shard5":{ "replicas":{ "core_node248":{ "node_name":"127.0.0.1:10040_solr", "core":"testNodeLost_shard5_replica_n247", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node50":{ "core":"testNodeLost_shard5_replica_n50", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10076_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node41":{ "core":"testNodeLost_shard5_replica_n41", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node43":{ "core":"testNodeLost_shard5_replica_n43", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node42":{ "core":"testNodeLost_shard5_replica_n42", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10089_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node45":{ "core":"testNodeLost_shard5_replica_n45", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node44":{ "core":"testNodeLost_shard5_replica_n44", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node47":{ "core":"testNodeLost_shard5_replica_n47", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node46":{ "core":"testNodeLost_shard5_replica_n46", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node49":{ "core":"testNodeLost_shard5_replica_n49", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"b3330000-bfffffff", "state":"active"}, "shard17":{ "replicas":{ "core_node169":{ "core":"testNodeLost_shard17_replica_n169", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node168":{ "core":"testNodeLost_shard17_replica_n168", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node167":{ "core":"testNodeLost_shard17_replica_n167", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node222":{ "node_name":"127.0.0.1:10002_solr", "core":"testNodeLost_shard17_replica_n221", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node165":{ "core":"testNodeLost_shard17_replica_n165", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node164":{ "core":"testNodeLost_shard17_replica_n164", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node163":{ "core":"testNodeLost_shard17_replica_n163", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node162":{ "core":"testNodeLost_shard17_replica_n162", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node161":{ "core":"testNodeLost_shard17_replica_n161", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node170":{ "core":"testNodeLost_shard17_replica_n170", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"4ccc0000-5998ffff", "state":"active"}, "shard18":{ "replicas":{ "core_node180":{ "core":"testNodeLost_shard18_replica_n180", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node179":{ "core":"testNodeLost_shard18_replica_n179", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node178":{ "core":"testNodeLost_shard18_replica_n178", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node177":{ "core":"testNodeLost_shard18_replica_n177", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node176":{ "core":"testNodeLost_shard18_replica_n176", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node175":{ "core":"testNodeLost_shard18_replica_n175", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node174":{ "core":"testNodeLost_shard18_replica_n174", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node173":{ "core":"testNodeLost_shard18_replica_n173", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node172":{ "core":"testNodeLost_shard18_replica_n172", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node171":{ "core":"testNodeLost_shard18_replica_n171", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"59990000-6665ffff", "state":"active"}, "shard19":{ "replicas":{ "core_node190":{ "core":"testNodeLost_shard19_replica_n190", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node226":{ "node_name":"127.0.0.1:10075_solr", "core":"testNodeLost_shard19_replica_n225", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node189":{ "core":"testNodeLost_shard19_replica_n189", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node188":{ "core":"testNodeLost_shard19_replica_n188", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node254":{ "node_name":"127.0.0.1:10030_solr", "core":"testNodeLost_shard19_replica_n253", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node187":{ "core":"testNodeLost_shard19_replica_n187", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node230":{ "node_name":"127.0.0.1:10024_solr", "core":"testNodeLost_shard19_replica_n229", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node184":{ "core":"testNodeLost_shard19_replica_n184", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node183":{ "core":"testNodeLost_shard19_replica_n183", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node181":{ "core":"testNodeLost_shard19_replica_n181", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"66660000-7332ffff", "state":"active"}, "shard1":{ "replicas":{ "core_node1":{ "core":"testNodeLost_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node234":{ "node_name":"127.0.0.1:10062_solr", "core":"testNodeLost_shard1_replica_n233", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node10":{ "core":"testNodeLost_shard1_replica_n10", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node7":{ "core":"testNodeLost_shard1_replica_n7", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node8":{ "core":"testNodeLost_shard1_replica_n8", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node9":{ "core":"testNodeLost_shard1_replica_n9", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{ "core":"testNodeLost_shard1_replica_n2", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node3":{ "core":"testNodeLost_shard1_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10057_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testNodeLost_shard1_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node5":{ "core":"testNodeLost_shard1_replica_n5", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10070_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"80000000-8ccbffff", "state":"active"}, "shard6":{ "replicas":{ "core_node204":{ "node_name":"127.0.0.1:10056_solr", "core":"testNodeLost_shard6_replica_n203", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node60":{ "core":"testNodeLost_shard6_replica_n60", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node51":{ "core":"testNodeLost_shard6_replica_n51", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node244":{ "node_name":"127.0.0.1:10046_solr", "core":"testNodeLost_shard6_replica_n243", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node53":{ "core":"testNodeLost_shard6_replica_n53", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node220":{ "node_name":"127.0.0.1:10097_solr", "core":"testNodeLost_shard6_replica_n219", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node56":{ "core":"testNodeLost_shard6_replica_n56", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node55":{ "core":"testNodeLost_shard6_replica_n55", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node240":{ "node_name":"127.0.0.1:10063_solr", "core":"testNodeLost_shard6_replica_n239", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node59":{ "core":"testNodeLost_shard6_replica_n59", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"c0000000-cccbffff", "state":"active"}, "shard7":{ "replicas":{ "core_node70":{ "core":"testNodeLost_shard7_replica_n70", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node61":{ "core":"testNodeLost_shard7_replica_n61", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node224":{ "node_name":"127.0.0.1:10078_solr", "core":"testNodeLost_shard7_replica_n223", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node63":{ "core":"testNodeLost_shard7_replica_n63", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node62":{ "core":"testNodeLost_shard7_replica_n62", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node65":{ "core":"testNodeLost_shard7_replica_n65", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node64":{ "core":"testNodeLost_shard7_replica_n64", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node67":{ "core":"testNodeLost_shard7_replica_n67", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node69":{ "core":"testNodeLost_shard7_replica_n69", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node68":{ "core":"testNodeLost_shard7_replica_n68", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"cccc0000-d998ffff", "state":"active"}, "shard8":{ "replicas":{ "core_node80":{ "core":"testNodeLost_shard8_replica_n80", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10013_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node72":{ "core":"testNodeLost_shard8_replica_n72", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10025_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node71":{ "core":"testNodeLost_shard8_replica_n71", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10088_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node74":{ "core":"testNodeLost_shard8_replica_n74", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node73":{ "core":"testNodeLost_shard8_replica_n73", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node76":{ "core":"testNodeLost_shard8_replica_n76", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node75":{ "core":"testNodeLost_shard8_replica_n75", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node78":{ "core":"testNodeLost_shard8_replica_n78", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node77":{ "core":"testNodeLost_shard8_replica_n77", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node79":{ "core":"testNodeLost_shard8_replica_n79", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10043_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"d9990000-e665ffff", "state":"active"}, "shard9":{ "replicas":{ "core_node90":{ "core":"testNodeLost_shard9_replica_n90", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10000_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node228":{ "node_name":"127.0.0.1:10050_solr", "core":"testNodeLost_shard9_replica_n227", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node81":{ "core":"testNodeLost_shard9_replica_n81", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node83":{ "core":"testNodeLost_shard9_replica_n83", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node256":{ "node_name":"127.0.0.1:10000_solr", "core":"testNodeLost_shard9_replica_n255", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node84":{ "core":"testNodeLost_shard9_replica_n84", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10095_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node87":{ "core":"testNodeLost_shard9_replica_n87", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10054_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node232":{ "node_name":"127.0.0.1:10008_solr", "core":"testNodeLost_shard9_replica_n231", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node89":{ "core":"testNodeLost_shard9_replica_n89", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node88":{ "core":"testNodeLost_shard9_replica_n88", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10003_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"e6660000-f332ffff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testNodeLost//clusterstate.json/93)={
"replicationFactor":"10",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"5",
"autoAddReplicas":"false",
"nrtReplicas":"10",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard20":{
"replicas":{
"core_node191":{
"core":"testNodeLost_shard20_replica_n191",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10011_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node258":{
"node_name":"127.0.0.1:10009_solr",
"core":"testNodeLost_shard20_replica_n257",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node200":{
"core":"testNodeLost_shard20_replica_n200",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10028_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node266":{
"node_name":"127.0.0.1:10028_solr",
"core":"testNodeLost_shard20_replica_n265",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"},
"core_node199":{
"core":"testNodeLost_shard20_replica_n199",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10074_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node196":{
"core":"testNodeLost_shard20_replica_n196",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10071_solr",
"state":"active",
"type":"NRT",
[...truncated too long message...]
d-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1287052585
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1287052585
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1287052585
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1656161918
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1656161918
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1656161918
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 64ms :: artifacts dl 9ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 275 minutes 48 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 61 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/61/
11 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:46031/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:37179/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:46031/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:37179/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([8448C61E6428EE4D:2E8515ECD3FB3B9D]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:288)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46031/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
... 46 more
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([8448C61E6428EE4D:C1CF9C4CAD483B5]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.MultiThreadedOCPTest.testFillWorkQueue(MultiThreadedOCPTest.java:113)
at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
Error Message:
should be at least one inactive event
Stack Trace:
java.lang.AssertionError: should be at least one inactive event
at __randomizedtesting.SeedInfo.seed([8448C61E6428EE4D:9964066C056BC946]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testBelowSearchRate
Error Message:
[Op{action=DELETEREPLICA, hints={COLL_SHARD=[{ "first":"belowRate_collection", "second":"shard1"}], REPLICA=[core_node6]}}, Op{action=DELETEREPLICA, hints={COLL_SHARD=[{ "first":"belowRate_collection", "second":"shard1"}], REPLICA=[core_node8]}}, Op{action=DELETEREPLICA, hints={COLL_SHARD=[{ "first":"belowRate_collection", "second":"shard1"}], REPLICA=[core_node10]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:37749_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:35431_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:38327_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:45021_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:34118_solr]}}] expected:<7> but was:<8>
Stack Trace:
java.lang.AssertionError: [Op{action=DELETEREPLICA, hints={COLL_SHARD=[{
"first":"belowRate_collection",
"second":"shard1"}], REPLICA=[core_node6]}}, Op{action=DELETEREPLICA, hints={COLL_SHARD=[{
"first":"belowRate_collection",
"second":"shard1"}], REPLICA=[core_node8]}}, Op{action=DELETEREPLICA, hints={COLL_SHARD=[{
"first":"belowRate_collection",
"second":"shard1"}], REPLICA=[core_node10]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:37749_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:35431_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:38327_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:45021_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:34118_solr]}}] expected:<7> but was:<8>
at __randomizedtesting.SeedInfo.seed([8448C61E6428EE4D:EEE5607E3CC06B5E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testBelowSearchRate(SearchRateTriggerIntegrationTest.java:394)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost
Error Message:
last state: DocCollection(testNodeLost//clusterstate.json/261)={ "replicationFactor":"10", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"5", "autoAddReplicas":"false", "nrtReplicas":"10", "tlogReplicas":"0", "autoCreated":"true", "shards":{ "shard20":{ "replicas":{ "core_node191":{ "core":"testNodeLost_shard20_replica_n191", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node200":{ "core":"testNodeLost_shard20_replica_n200", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node199":{ "core":"testNodeLost_shard20_replica_n199", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10121_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node198":{ "core":"testNodeLost_shard20_replica_n198", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10117_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node197":{ "core":"testNodeLost_shard20_replica_n197", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node196":{ "core":"testNodeLost_shard20_replica_n196", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10101_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node195":{ "core":"testNodeLost_shard20_replica_n195", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node194":{ "core":"testNodeLost_shard20_replica_n194", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node193":{ "core":"testNodeLost_shard20_replica_n193", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node192":{ "core":"testNodeLost_shard20_replica_n192", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10126_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"73330000-7fffffff", "state":"active"}, "shard10":{ "replicas":{ "core_node92":{ "core":"testNodeLost_shard10_replica_n92", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10126_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node91":{ "core":"testNodeLost_shard10_replica_n91", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10081_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node94":{ "core":"testNodeLost_shard10_replica_n94", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10026_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node93":{ "core":"testNodeLost_shard10_replica_n93", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10098_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node96":{ "core":"testNodeLost_shard10_replica_n96", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10101_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node95":{ "core":"testNodeLost_shard10_replica_n95", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10016_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node98":{ "core":"testNodeLost_shard10_replica_n98", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10117_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node100":{ "core":"testNodeLost_shard10_replica_n100", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10015_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node97":{ "core":"testNodeLost_shard10_replica_n97", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10037_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node99":{ "core":"testNodeLost_shard10_replica_n99", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10121_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"f3330000-ffffffff", "state":"active"}, "shard11":{ "replicas":{ "core_node108":{ "core":"testNodeLost_shard11_replica_n108", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node107":{ "core":"testNodeLost_shard11_replica_n107", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node106":{ "core":"testNodeLost_shard11_replica_n106", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node105":{ "core":"testNodeLost_shard11_replica_n105", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10021_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node104":{ "core":"testNodeLost_shard11_replica_n104", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node103":{ "core":"testNodeLost_shard11_replica_n103", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node102":{ "core":"testNodeLost_shard11_replica_n102", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node101":{ "core":"testNodeLost_shard11_replica_n101", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node110":{ "core":"testNodeLost_shard11_replica_n110", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10103_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node109":{ "core":"testNodeLost_shard11_replica_n109", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10132_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"0-ccbffff", "state":"active"}, "shard12":{ "replicas":{ "core_node119":{ "core":"testNodeLost_shard12_replica_n119", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node118":{ "core":"testNodeLost_shard12_replica_n118", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10106_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node117":{ "core":"testNodeLost_shard12_replica_n117", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10102_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node116":{ "core":"testNodeLost_shard12_replica_n116", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10096_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node115":{ "core":"testNodeLost_shard12_replica_n115", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node114":{ "core":"testNodeLost_shard12_replica_n114", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node113":{ "core":"testNodeLost_shard12_replica_n113", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node112":{ "core":"testNodeLost_shard12_replica_n112", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node111":{ "core":"testNodeLost_shard12_replica_n111", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node120":{ "core":"testNodeLost_shard12_replica_n120", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"ccc0000-1998ffff", "state":"active"}, "shard13":{ "replicas":{ "core_node129":{ "core":"testNodeLost_shard13_replica_n129", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node206":{ "node_name":"127.0.0.1:10103_solr", "core":"testNodeLost_shard13_replica_n205", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node128":{ "core":"testNodeLost_shard13_replica_n128", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node126":{ "core":"testNodeLost_shard13_replica_n126", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node125":{ "core":"testNodeLost_shard13_replica_n125", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node124":{ "core":"testNodeLost_shard13_replica_n124", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node123":{ "core":"testNodeLost_shard13_replica_n123", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10115_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node122":{ "core":"testNodeLost_shard13_replica_n122", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node121":{ "core":"testNodeLost_shard13_replica_n121", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node130":{ "core":"testNodeLost_shard13_replica_n130", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"19990000-2665ffff", "state":"active"}, "shard2":{ "replicas":{ "core_node18":{ "core":"testNodeLost_shard2_replica_n18", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10106_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node17":{ "core":"testNodeLost_shard2_replica_n17", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10102_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node19":{ "core":"testNodeLost_shard2_replica_n19", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10024_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node20":{ "core":"testNodeLost_shard2_replica_n20", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10012_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node12":{ "core":"testNodeLost_shard2_replica_n12", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10072_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node11":{ "core":"testNodeLost_shard2_replica_n11", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10041_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node14":{ "core":"testNodeLost_shard2_replica_n14", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10020_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node13":{ "core":"testNodeLost_shard2_replica_n13", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10052_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node16":{ "core":"testNodeLost_shard2_replica_n16", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10096_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node15":{ "core":"testNodeLost_shard2_replica_n15", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10046_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"8ccc0000-9998ffff", "state":"active"}, "shard3":{ "replicas":{ "core_node29":{ "core":"testNodeLost_shard3_replica_n29", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10053_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node28":{ "core":"testNodeLost_shard3_replica_n28", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10099_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node30":{ "core":"testNodeLost_shard3_replica_n30", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10029_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node21":{ "core":"testNodeLost_shard3_replica_n21", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10045_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node23":{ "core":"testNodeLost_shard3_replica_n23", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10115_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node22":{ "core":"testNodeLost_shard3_replica_n22", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10073_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node25":{ "core":"testNodeLost_shard3_replica_n25", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10093_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node24":{ "core":"testNodeLost_shard3_replica_n24", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10087_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node26":{ "core":"testNodeLost_shard3_replica_n26", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10062_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node208":{ "node_name":"127.0.0.1:10059_solr", "core":"testNodeLost_shard3_replica_n207", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}}, "range":"99990000-a665ffff", "state":"active"}, "shard14":{ "replicas":{ "core_node139":{ "core":"testNodeLost_shard14_replica_n139", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node138":{ "core":"testNodeLost_shard14_replica_n138", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10127_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node137":{ "core":"testNodeLost_shard14_replica_n137", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10110_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node136":{ "core":"testNodeLost_shard14_replica_n136", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node135":{ "core":"testNodeLost_shard14_replica_n135", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10129_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node134":{ "core":"testNodeLost_shard14_replica_n134", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10100_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node133":{ "core":"testNodeLost_shard14_replica_n133", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10119_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node132":{ "core":"testNodeLost_shard14_replica_n132", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node131":{ "core":"testNodeLost_shard14_replica_n131", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node140":{ "core":"testNodeLost_shard14_replica_n140", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"26660000-3332ffff", "state":"active"}, "shard15":{ "replicas":{ "core_node149":{ "core":"testNodeLost_shard15_replica_n149", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node148":{ "core":"testNodeLost_shard15_replica_n148", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node147":{ "core":"testNodeLost_shard15_replica_n147", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10133_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node146":{ "core":"testNodeLost_shard15_replica_n146", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node145":{ "core":"testNodeLost_shard15_replica_n145", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node144":{ "core":"testNodeLost_shard15_replica_n144", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10116_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node143":{ "core":"testNodeLost_shard15_replica_n143", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node142":{ "core":"testNodeLost_shard15_replica_n142", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node141":{ "core":"testNodeLost_shard15_replica_n141", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node150":{ "core":"testNodeLost_shard15_replica_n150", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"33330000-3fffffff", "state":"active"}, "shard4":{ "replicas":{ "core_node39":{ "core":"testNodeLost_shard4_replica_n39", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10090_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node40":{ "core":"testNodeLost_shard4_replica_n40", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10085_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node32":{ "core":"testNodeLost_shard4_replica_n32", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10056_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node31":{ "core":"testNodeLost_shard4_replica_n31", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10018_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node34":{ "core":"testNodeLost_shard4_replica_n34", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10100_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node33":{ "core":"testNodeLost_shard4_replica_n33", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10119_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node36":{ "core":"testNodeLost_shard4_replica_n36", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10067_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node35":{ "core":"testNodeLost_shard4_replica_n35", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10129_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node38":{ "core":"testNodeLost_shard4_replica_n38", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10127_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node37":{ "core":"testNodeLost_shard4_replica_n37", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10110_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"a6660000-b332ffff", "state":"active"}, "shard5":{ "replicas":{ "core_node50":{ "core":"testNodeLost_shard5_replica_n50", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10078_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node41":{ "core":"testNodeLost_shard5_replica_n41", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10006_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node43":{ "core":"testNodeLost_shard5_replica_n43", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10094_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node42":{ "core":"testNodeLost_shard5_replica_n42", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node45":{ "core":"testNodeLost_shard5_replica_n45", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10064_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node44":{ "core":"testNodeLost_shard5_replica_n44", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10116_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node47":{ "core":"testNodeLost_shard5_replica_n47", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10133_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node46":{ "core":"testNodeLost_shard5_replica_n46", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node49":{ "core":"testNodeLost_shard5_replica_n49", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10086_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node48":{ "core":"testNodeLost_shard5_replica_n48", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10048_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"b3330000-bfffffff", "state":"active"}, "shard16":{ "replicas":{ "core_node159":{ "core":"testNodeLost_shard16_replica_n159", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10131_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node158":{ "core":"testNodeLost_shard16_replica_n158", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node157":{ "core":"testNodeLost_shard16_replica_n157", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10105_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node156":{ "core":"testNodeLost_shard16_replica_n156", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node155":{ "core":"testNodeLost_shard16_replica_n155", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node154":{ "core":"testNodeLost_shard16_replica_n154", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10123_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node153":{ "core":"testNodeLost_shard16_replica_n153", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10120_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node152":{ "core":"testNodeLost_shard16_replica_n152", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node151":{ "core":"testNodeLost_shard16_replica_n151", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10122_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node160":{ "core":"testNodeLost_shard16_replica_n160", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"40000000-4ccbffff", "state":"active"}, "shard17":{ "replicas":{ "core_node169":{ "core":"testNodeLost_shard17_replica_n169", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node168":{ "core":"testNodeLost_shard17_replica_n168", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node167":{ "core":"testNodeLost_shard17_replica_n167", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10111_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node166":{ "core":"testNodeLost_shard17_replica_n166", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node165":{ "core":"testNodeLost_shard17_replica_n165", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10125_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node164":{ "core":"testNodeLost_shard17_replica_n164", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node163":{ "core":"testNodeLost_shard17_replica_n163", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node162":{ "core":"testNodeLost_shard17_replica_n162", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node161":{ "core":"testNodeLost_shard17_replica_n161", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node170":{ "core":"testNodeLost_shard17_replica_n170", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"4ccc0000-5998ffff", "state":"active"}, "shard18":{ "replicas":{ "core_node180":{ "core":"testNodeLost_shard18_replica_n180", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node179":{ "core":"testNodeLost_shard18_replica_n179", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node178":{ "core":"testNodeLost_shard18_replica_n178", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node177":{ "core":"testNodeLost_shard18_replica_n177", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node176":{ "core":"testNodeLost_shard18_replica_n176", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10108_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node175":{ "core":"testNodeLost_shard18_replica_n175", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node174":{ "core":"testNodeLost_shard18_replica_n174", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node173":{ "core":"testNodeLost_shard18_replica_n173", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node172":{ "core":"testNodeLost_shard18_replica_n172", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10112_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node171":{ "core":"testNodeLost_shard18_replica_n171", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"59990000-6665ffff", "state":"active"}, "shard19":{ "replicas":{ "core_node190":{ "core":"testNodeLost_shard19_replica_n190", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node202":{ "node_name":"127.0.0.1:10072_solr", "core":"testNodeLost_shard19_replica_n201", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node189":{ "core":"testNodeLost_shard19_replica_n189", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node188":{ "core":"testNodeLost_shard19_replica_n188", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node187":{ "core":"testNodeLost_shard19_replica_n187", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10114_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node186":{ "core":"testNodeLost_shard19_replica_n186", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node184":{ "core":"testNodeLost_shard19_replica_n184", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node183":{ "core":"testNodeLost_shard19_replica_n183", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node182":{ "core":"testNodeLost_shard19_replica_n182", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node181":{ "core":"testNodeLost_shard19_replica_n181", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"66660000-7332ffff", "state":"active"}, "shard1":{ "replicas":{ "core_node1":{ "core":"testNodeLost_shard1_replica_n1", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10038_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node6":{ "core":"testNodeLost_shard1_replica_n6", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10050_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node10":{ "core":"testNodeLost_shard1_replica_n10", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10103_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node7":{ "core":"testNodeLost_shard1_replica_n7", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10059_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node8":{ "core":"testNodeLost_shard1_replica_n8", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10084_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node9":{ "core":"testNodeLost_shard1_replica_n9", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10132_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{ "core":"testNodeLost_shard1_replica_n2", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10027_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node3":{ "core":"testNodeLost_shard1_replica_n3", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10055_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node4":{ "core":"testNodeLost_shard1_replica_n4", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10034_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node5":{ "core":"testNodeLost_shard1_replica_n5", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10021_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"80000000-8ccbffff", "state":"active"}, "shard6":{ "replicas":{ "core_node60":{ "core":"testNodeLost_shard6_replica_n60", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10023_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node52":{ "core":"testNodeLost_shard6_replica_n52", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10066_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node51":{ "core":"testNodeLost_shard6_replica_n51", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10122_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node54":{ "core":"testNodeLost_shard6_replica_n54", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10123_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node53":{ "core":"testNodeLost_shard6_replica_n53", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10120_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node56":{ "core":"testNodeLost_shard6_replica_n56", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10079_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node55":{ "core":"testNodeLost_shard6_replica_n55", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10030_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node58":{ "core":"testNodeLost_shard6_replica_n58", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10031_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node57":{ "core":"testNodeLost_shard6_replica_n57", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10105_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node59":{ "core":"testNodeLost_shard6_replica_n59", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10131_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"c0000000-cccbffff", "state":"active"}, "shard7":{ "replicas":{ "core_node70":{ "core":"testNodeLost_shard7_replica_n70", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10091_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node61":{ "core":"testNodeLost_shard7_replica_n61", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10051_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node63":{ "core":"testNodeLost_shard7_replica_n63", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10033_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node62":{ "core":"testNodeLost_shard7_replica_n62", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10044_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node65":{ "core":"testNodeLost_shard7_replica_n65", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10125_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node64":{ "core":"testNodeLost_shard7_replica_n64", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10047_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node67":{ "core":"testNodeLost_shard7_replica_n67", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10111_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node66":{ "core":"testNodeLost_shard7_replica_n66", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10080_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node69":{ "core":"testNodeLost_shard7_replica_n69", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10039_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node68":{ "core":"testNodeLost_shard7_replica_n68", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10019_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"cccc0000-d998ffff", "state":"active"}, "shard8":{ "replicas":{ "core_node80":{ "core":"testNodeLost_shard8_replica_n80", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10077_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node72":{ "core":"testNodeLost_shard8_replica_n72", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10112_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node71":{ "core":"testNodeLost_shard8_replica_n71", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10092_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node74":{ "core":"testNodeLost_shard8_replica_n74", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10060_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node73":{ "core":"testNodeLost_shard8_replica_n73", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10036_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node76":{ "core":"testNodeLost_shard8_replica_n76", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10108_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node75":{ "core":"testNodeLost_shard8_replica_n75", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10010_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node78":{ "core":"testNodeLost_shard8_replica_n78", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10068_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node77":{ "core":"testNodeLost_shard8_replica_n77", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10097_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node79":{ "core":"testNodeLost_shard8_replica_n79", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10040_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"d9990000-e665ffff", "state":"active"}, "shard9":{ "replicas":{ "core_node90":{ "core":"testNodeLost_shard9_replica_n90", "leader":"true", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10042_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node81":{ "core":"testNodeLost_shard9_replica_n81", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10061_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node204":{ "node_name":"127.0.0.1:10056_solr", "core":"testNodeLost_shard9_replica_n203", "state":"active", "INDEX.sizeInBytes":1, "type":"NRT"}, "core_node83":{ "core":"testNodeLost_shard9_replica_n83", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node82":{ "core":"testNodeLost_shard9_replica_n82", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10017_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node84":{ "core":"testNodeLost_shard9_replica_n84", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10035_solr", "state":"down", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node87":{ "core":"testNodeLost_shard9_replica_n87", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10114_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node86":{ "core":"testNodeLost_shard9_replica_n86", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10011_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node89":{ "core":"testNodeLost_shard9_replica_n89", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10075_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node88":{ "core":"testNodeLost_shard9_replica_n88", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10058_solr", "state":"active", "type":"NRT", "SEARCHER.searcher.numDocs":0}}, "range":"e6660000-f332ffff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testNodeLost//clusterstate.json/261)={
"replicationFactor":"10",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"5",
"autoAddReplicas":"
[...truncated too long message...]
figure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null699932580
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null699932580
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null699932580
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null454020974
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null454020974
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null454020974
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 57ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 308 minutes 37 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 60 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/60/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([AAB60BDC2D23D53B:F7FE1555E2E57374]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:579)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
there should be new MOVERPLICA ops
Stack Trace:
java.lang.AssertionError: there should be new MOVERPLICA ops
at __randomizedtesting.SeedInfo.seed([AAB60BDC2D23D53B:14C16C9F2FF5315]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13424 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> 3208706 INFO (SUITE-TestLargeCluster-seed#[AAB60BDC2D23D53B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_AAB60BDC2D23D53B-001/init-core-data-001
[junit4] 2> 3208719 WARN (SUITE-TestLargeCluster-seed#[AAB60BDC2D23D53B]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 3208719 INFO (SUITE-TestLargeCluster-seed#[AAB60BDC2D23D53B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 3208721 INFO (SUITE-TestLargeCluster-seed#[AAB60BDC2D23D53B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3208722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3208722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 3208723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 3208723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 3208729 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10043_solr, 127.0.0.1:10018_solr, 127.0.0.1:10057_solr, 127.0.0.1:10049_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10005_solr, 127.0.0.1:10013_solr, 127.0.0.1:10021_solr, 127.0.0.1:10058_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10022_solr, 127.0.0.1:10039_solr, 127.0.0.1:10033_solr, 127.0.0.1:10025_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10044_solr, 127.0.0.1:10081_solr, 127.0.0.1:10028_solr, 127.0.0.1:10036_solr, 127.0.0.1:10084_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10041_solr, 127.0.0.1:10079_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10054_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10015_solr, 127.0.0.1:10055_solr, 127.0.0.1:10069_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10062_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10045_solr, 127.0.0.1:10037_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10053_solr, 127.0.0.1:10083_solr, 127.0.0.1:10059_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10056_solr, 127.0.0.1:10064_solr, 127.0.0.1:10072_solr]
[junit4] 2> 3208730 DEBUG (ScheduledTrigger-5640-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 85
[junit4] 2> 3208730 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3208730 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 3208751 DEBUG (ScheduledTrigger-5640-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208761 INFO (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
[junit4] 2> 3208761 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3208761 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3208761 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3208761 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3208761 INFO (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3208761 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3208761 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2034) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 3208766 DEBUG (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3208766 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3208767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3208767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 3208767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3208767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 3208767 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208767 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3208768 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3208797 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208817 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208837 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208857 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208869 DEBUG (simCloudManagerPool-5642-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3208869 DEBUG (simCloudManagerPool-5642-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3208869 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3208870 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3208870 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3208873 DEBUG (simCloudManagerPool-5642-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3208873 DEBUG (simCloudManagerPool-5642-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3208873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3208873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 3208873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3208873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3208885 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208885 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3208906 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208906 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3208986 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3208986 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209016 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209016 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209052 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209052 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209089 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209089 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209109 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209109 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209129 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209129 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209149 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209149 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209170 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209170 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209190 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209190 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209210 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209210 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209230 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209231 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209251 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209251 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209271 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209271 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209291 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209291 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209312 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209312 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209332 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209332 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209352 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209352 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209372 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209372 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209393 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209393 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209413 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209413 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209433 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209433 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209453 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209453 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209474 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209474 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209494 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209494 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209514 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209514 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209535 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209535 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209555 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209555 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209575 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209575 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209595 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209595 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209616 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209616 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209636 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209636 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209656 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209656 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209676 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209676 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209697 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209697 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209717 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209717 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209737 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209737 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209757 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209757 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209778 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209778 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209798 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209798 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209818 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209818 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209839 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209839 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209859 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209859 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209879 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209879 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209899 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209900 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209920 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209920 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209940 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209940 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209960 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209960 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3209981 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3209981 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210001 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210001 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210021 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210021 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210041 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210042 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210062 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210062 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210092 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210092 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210112 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210112 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210133 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210133 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210153 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210153 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210173 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210173 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210193 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210193 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210214 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210214 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210234 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210234 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210254 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210254 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210274 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210274 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210295 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210295 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210315 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210315 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210335 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210335 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210356 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210356 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210381 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210381 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210401 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210401 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210421 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210421 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210441 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210441 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210462 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210462 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210482 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210482 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210502 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210502 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210522 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210523 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210543 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210543 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210567 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210567 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210588 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210588 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210608 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210608 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210641 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210641 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210661 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210661 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210697 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210697 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210718 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210718 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210742 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210742 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210762 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210762 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210785 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210785 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210805 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210805 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210825 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210825 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210846 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210846 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210866 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210866 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210886 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210886 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210917 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210917 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210941 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210941 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210961 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210961 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3210982 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3210982 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211006 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211006 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211043 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211043 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211063 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211063 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211095 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211096 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211116 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211116 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211152 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211153 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211189 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211189 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211225 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211226 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211246 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211246 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211266 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211266 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211286 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211286 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211307 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211307 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211327 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211327 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211342 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard20: {"core_node191":{
[junit4] 2> "core":"testNodeLost_shard20_replica_n191",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10025_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard20",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211342 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard10: {"core_node92":{
[junit4] 2> "core":"testNodeLost_shard10_replica_n92",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10102_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard10",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211342 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard11: {"core_node108":{
[junit4] 2> "core":"testNodeLost_shard11_replica_n108",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10058_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard11",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard12: {"core_node119":{
[junit4] 2> "core":"testNodeLost_shard12_replica_n119",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10099_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard12",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard2: {"core_node18":{
[junit4] 2> "core":"testNodeLost_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10092_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard13: {"core_node129":{
[junit4] 2> "core":"testNodeLost_shard13_replica_n129",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10060_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard13",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard3: {"core_node29":{
[junit4] 2> "core":"testNodeLost_shard3_replica_n29",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10060_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard14: {"core_node139":{
[junit4] 2> "core":"testNodeLost_shard14_replica_n139",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10096_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard14",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211343 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard15: {"core_node149":{
[junit4] 2> "core":"testNodeLost_shard15_replica_n149",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10009_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard15",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211364 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard4: {"core_node39":{
[junit4] 2> "core":"testNodeLost_shard4_replica_n39",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10096_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard17: {"core_node169":{
[junit4] 2> "core":"testNodeLost_shard17_replica_n169",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard17",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard18: {"core_node180":{
[junit4] 2> "core":"testNodeLost_shard18_replica_n180",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10018_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard18",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard19: {"core_node190":{
[junit4] 2> "core":"testNodeLost_shard19_replica_n190",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10090_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard19",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard1: {"core_node1":{
[junit4] 2> "core":"testNodeLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10069_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard6: {"core_node60":{
[junit4] 2> "core":"testNodeLost_shard6_replica_n60",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10064_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard6",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard7: {"core_node70":{
[junit4] 2> "core":"testNodeLost_shard7_replica_n70",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10067_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard7",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard8: {"core_node80":{
[junit4] 2> "core":"testNodeLost_shard8_replica_n80",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10018_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard8",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211426 DEBUG (simCloudManagerPool-5642-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard9: {"core_node90":{
[junit4] 2> "core":"testNodeLost_shard9_replica_n90",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10090_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard9",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211425 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211425 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211444 DEBUG (simCloudManagerPool-5642-thread-200) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard5: {"core_node50":{
[junit4] 2> "core":"testNodeLost_shard5_replica_n50",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10012_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211445 DEBUG (simCloudManagerPool-5642-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard16: {"core_node159":{
[junit4] 2> "core":"testNodeLost_shard16_replica_n159",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10022_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard16",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211461 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211462 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211486 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 3211486 DEBUG (ScheduledTrigger-5643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 3211519 INFO (TEST-TestLargeCluster.testNodeLost-seed#[AAB60BDC2D23D53B]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 4922 ms
[junit4] 2> 3211526 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 3211526 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10082_solr
[junit4] 2> 3211526 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 99
[junit4] 2> 3211526 DEBUG (ScheduledTrigger-5643-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10082_solr
[junit4] 2> 3211555 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 3211571 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 99
[junit4] 2> 3211579 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 3211591 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 99
[junit4] 2> 3211599 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 3211611 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 99
[junit4] 2> 3211619 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 99
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 98
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10018_solr
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 98
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10018_solr
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10082_solr]
[junit4] 2> 3211640 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"1fd70f9e0196a3Tcbxuq1csavgppe1ywi2v9qqat",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":8962186353415843,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[8962186353415843],
[junit4] 2> "nodeNames":["127.0.0.1:10082_solr"]}}
[junit4] 2> 3211661 DEBUG (ScheduledTrigger-5643-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 3211662 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"1fd70f9e0196a3Tcbxuq1csavgppe1ywi2v9qqat",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":8962186353415843,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[8962186353415843],
[junit4] 2> "_enqueue_time_":8962193099886693,
[junit4] 2> "nodeNames":["127.0.0.1:10082_solr"]}}
[junit4] 2> 3211662 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"1fd70f9e0196a3Tcbxuq1csavgppe1ywi2v9qqat",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":8962186353415843,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[8962186353415843],
[junit4] 2> "_enqueue_time_":8962193099886693,
[junit4] 2> "nodeNames":["127.0.0.1:10082_solr"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 3211678 DEBUG (simCloudManagerPool-5642-thread-41) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard18: {"core_node179":{
[junit4] 2> "core":"testNodeLost_shard18_replica_n179",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10011_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard18",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3211678 DEBUG (simCloudManagerPool-5642-thread-43) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard8: {"core_node72":{
[junit4] 2> "core":"testNodeLost_shard8_replica_n72",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10029_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard8",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3212180 INFO (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node189
[junit4] 2> 3212310 INFO (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node89
[junit4] 2> 3212310 INFO (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 8962225576 sessionWrapper.createTime 8962194962787293, this.sessionWrapper.createTime 8962194962787293
[junit4] 2> 3212310 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"1fd70f9e0196a3Tcbxuq1csavgppe1ywi2v9qqat",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":8962186353415843,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[8962186353415843],
[junit4] 2> "_enqueue_time_":8962193099886693,
[junit4] 2> "nodeNames":["127.0.0.1:10082_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@d64494e, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@3ae769b1], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 3212311 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node189
[junit4] 2> 3212413 DEBUG (simCloudManagerPool-5642-thread-185) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node202":{
[junit4] 2> "core":"testNodeLost_shard19_replica_n201",
[junit4] 2> "shard":"shard19",
[junit4] 2> "collection":"testNodeLost",
[junit4] 2> "node_name":"127.0.0.1:10072_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 3212539 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node89
[junit4] 2> 3212647 DEBUG (simCloudManagerPool-5642-thread-200) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node204":{
[junit4] 2> "core":"testNodeLost_shard9_replica_n203",
[junit4] 2> "shard":"shard9",
[junit4] 2> "collection":"testNodeLost",
[junit4] 2> "node_name":"127.0.0.1:10056_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 3212863 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3212863 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 3212863 DEBUG (AutoscalingActionExecutor-5644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 60044 ms for event id=1fd70f9e0196a3Tcbxuq1csavgppe1ywi2v9qqat
[junit4] 2> 3212945 DEBUG (simCloudManagerPool-5642-thread-111) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard14: {"core_node138":{
[junit4] 2> "core":"testNodeLost_shard14_replica_n138",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10055_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard14",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3212946 DEBUG (simCloudManagerPool-5642-thread-111) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard4: {"core_node40":{
[junit4] 2> "core":"testNodeLost_shard4_replica_n40",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":1,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testNodeLost"}}
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 87
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10049_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10104_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10020_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10065_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10021_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10066_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10035_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10051_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
[junit4] 2> 3212972 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10096_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 87
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10049_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10104_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10020_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10065_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10021_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10066_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10035_solr
[junit4] 2> 3212973 DEBUG (ScheduledTrigger-5643-thr
[...truncated too long message...]
figure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1605961234
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1605961234
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1605961234
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null33474232
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null33474232
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null33474232
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 51ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 305 minutes 51 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 59 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/59/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [Overseer] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.Overseer.start(Overseer.java:545) at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135) at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307) at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393) at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2081) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [Overseer]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.cloud.Overseer.start(Overseer.java:545)
at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2081)
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C328FE39FAA33E5B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:303)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple2 null Live Nodes: [127.0.0.1:44173_solr, 127.0.0.1:58743_solr] Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/25)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node2":{ "dataDir":"hdfs://localhost:35226/data/testSimple2/core_node2/data/", "base_url":"http://127.0.0.1:44173/solr", "node_name":"127.0.0.1:44173_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node2/data/tlog", "core":"testSimple2_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node5":{ "dataDir":"hdfs://localhost:35226/data/testSimple2/core_node5/data/", "base_url":"http://127.0.0.1:36725/solr", "node_name":"127.0.0.1:36725_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node5/data/tlog", "core":"testSimple2_shard1_replica_n3", "shared_storage":"true", "state":"down"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://localhost:35226/data/testSimple2/core_node7/data/", "base_url":"http://127.0.0.1:44173/solr", "node_name":"127.0.0.1:44173_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node7/data/tlog", "core":"testSimple2_shard2_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://localhost:35226/data/testSimple2/core_node8/data/", "base_url":"http://127.0.0.1:36725/solr", "node_name":"127.0.0.1:36725_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node8/data/tlog", "core":"testSimple2_shard2_replica_n6", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple2
null
Live Nodes: [127.0.0.1:44173_solr, 127.0.0.1:58743_solr]
Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/25)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node2":{
"dataDir":"hdfs://localhost:35226/data/testSimple2/core_node2/data/",
"base_url":"http://127.0.0.1:44173/solr",
"node_name":"127.0.0.1:44173_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node2/data/tlog",
"core":"testSimple2_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node5":{
"dataDir":"hdfs://localhost:35226/data/testSimple2/core_node5/data/",
"base_url":"http://127.0.0.1:36725/solr",
"node_name":"127.0.0.1:36725_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node5/data/tlog",
"core":"testSimple2_shard1_replica_n3",
"shared_storage":"true",
"state":"down"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://localhost:35226/data/testSimple2/core_node7/data/",
"base_url":"http://127.0.0.1:44173/solr",
"node_name":"127.0.0.1:44173_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node7/data/tlog",
"core":"testSimple2_shard2_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://localhost:35226/data/testSimple2/core_node8/data/",
"base_url":"http://127.0.0.1:36725/solr",
"node_name":"127.0.0.1:36725_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://localhost:35226/data/testSimple2/core_node8/data/tlog",
"core":"testSimple2_shard2_replica_n6",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([C328FE39FAA33E5B:FB9BDAC7DD50EA8A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration
Error Message:
did not finish processing in time
Stack Trace:
java.lang.AssertionError: did not finish processing in time
at __randomizedtesting.SeedInfo.seed([C328FE39FAA33E5B:9091BC8918B2ABA1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:404)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([C328FE39FAA33E5B]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([C328FE39FAA33E5B]:0)
Build Log:
[...truncated 12851 lines...]
[junit4] Suite: org.apache.solr.cloud.ZkControllerTest
[junit4] 2> 888857 INFO (SUITE-ZkControllerTest-seed#[C328FE39FAA33E5B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.ZkControllerTest_C328FE39FAA33E5B-001/init-core-data-001
[junit4] 2> 888857 WARN (SUITE-ZkControllerTest-seed#[C328FE39FAA33E5B]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=214 numCloses=214
[junit4] 2> 888868 INFO (SUITE-ZkControllerTest-seed#[C328FE39FAA33E5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 888870 INFO (SUITE-ZkControllerTest-seed#[C328FE39FAA33E5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 888873 INFO (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeNameUrlConversion
[junit4] 2> 888874 INFO (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 888885 INFO (Thread-882) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 888885 INFO (Thread-882) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 888912 ERROR (Thread-882) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 888985 INFO (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.ZkTestServer start zk server on port:43690
[junit4] 2> 889216 INFO (zkConnectionManagerCallback-1370-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889244 INFO (zkConnectionManagerCallback-1372-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889268 INFO (zkConnectionManagerCallback-1374-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889408 ERROR (zkCallback-1373-thread-1) [ ] o.a.s.c.c.ZkStateReader Error reading cluster properties from zookeeper
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterprops.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.loadClusterProperties(ZkStateReader.java:989) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$new$5(ZkStateReader.java:981) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 889416 ERROR (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 889416 INFO (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43690 43690
[junit4] 2> 889518 INFO (Thread-882) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43690 43690
[junit4] 2> 889519 WARN (Thread-882) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 889520 INFO (TEST-ZkControllerTest.testNodeNameUrlConversion-seed#[C328FE39FAA33E5B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeNameUrlConversion
[junit4] 2> 889522 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReadConfigName
[junit4] 2> 889522 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 889522 INFO (Thread-883) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 889522 INFO (Thread-883) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 889530 ERROR (Thread-883) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 889676 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.ZkTestServer start zk server on port:38491
[junit4] 2> 889736 INFO (zkConnectionManagerCallback-1380-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889756 INFO (zkConnectionManagerCallback-1382-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889791 INFO (zkConnectionManagerCallback-1384-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889801 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@537b6faa, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 889845 INFO (zkConnectionManagerCallback-1388-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 889995 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 889995 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:8983_solr
[junit4] 2> 889996 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.Overseer Overseer (id=73802051630596099-127.0.0.1:8983_solr-n_0000000000) starting
[junit4] 2> 890096 INFO (zkConnectionManagerCallback-1395-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 890098 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38491/solr ready
[junit4] 2> 890123 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:8983_solr
[junit4] 2> 890132 INFO (OverseerStateUpdate-73802051630596099-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 890134 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.Overseer Overseer (id=73802051630596099-127.0.0.1:8983_solr-n_0000000000) closing
[junit4] 2> 890156 INFO (zkCallback-1394-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 890156 INFO (OverseerStateUpdate-73802051630596099-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:8983_solr
[junit4] 2> 890185 WARN (OverseerExitThread) [ ] o.a.s.c.Overseer I'm exiting, but I'm still the leader
[junit4] 2> 890233 WARN (OverseerAutoScalingTriggerThread-73802051630596099-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting.
[junit4] 2> 890234 WARN (ScheduledTrigger-1787-thread-1) [ ] o.a.s.c.a.TriggerBase Exception updating trigger state '/autoscaling/triggerState/.auto_add_replicas'
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.lang.Object.wait(Native Method) ~[?:1.8.0_172]
[junit4] 2> at java.lang.Object.wait(Object.java:502) ~[?:1.8.0_172]
[junit4] 2> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1409) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1100) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316) ~[java/:?]
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerBase.saveState(TriggerBase.java:234) [java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:596) [java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 890237 INFO (OverseerExitThread) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:8983_solr
[junit4] 2> 890238 INFO (OverseerExitThread) [ ] o.a.s.c.Overseer Overseer (id=73802051630596099-127.0.0.1:8983_solr-n_0000000001) starting
[junit4] 2> 890277 WARN (OverseerAutoScalingTriggerThread-73802051630596099-127.0.0.1:8983_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Solr cannot talk to ZK, exiting OverseerTriggerThread main queue loop
[junit4] 2> java.io.IOException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:183) ~[java/:?]
[junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:83) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:131) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) ~[java/:?]
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:176) ~[java/:?]
[junit4] 2> ... 3 more
[junit4] 2> 890280 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=605559004
[junit4] 2> 890280 ERROR (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.SolrCores No transient handler has been defined. Check solr.xml to see if an attempt to provide a custom TransientSolrCoreCacheFactory was done incorrectly since the default should have been used otherwise.
[junit4] 2> 890280 ERROR (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 890281 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38491 38491
[junit4] 2> 890281 ERROR (OverseerCollectionConfigSetProcessor-73802051630596099-127.0.0.1:8983_solr-n_0000000001) [ ] o.a.s.c.OverseerTaskProcessor :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/collection-queue-work
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:410)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskQueue.getTailId(OverseerTaskQueue.java:272)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 890282 INFO (OverseerCollectionConfigSetProcessor-73802051630596099-127.0.0.1:8983_solr-n_0000000001) [ ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=73802051630596099-127.0.0.1:8983_solr-n_0000000001) am no longer a leader.
[junit4] 2> 890377 INFO (Thread-883) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38491 38491
[junit4] 2> 890377 WARN (Thread-883) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 890378 INFO (TEST-ZkControllerTest.testReadConfigName-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.SolrTestCaseJ4 ###Ending testReadConfigName
[junit4] 2> 890380 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.SolrTestCaseJ4 ###Starting testPublishAndWaitForDownStates
[junit4] 2> 890381 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 890388 INFO (Thread-884) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 890388 INFO (Thread-884) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 890412 ERROR (Thread-884) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 890488 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkTestServer start zk server on port:44338
[junit4] 2> 890497 INFO (zkConnectionManagerCallback-1397-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 890499 INFO (zkConnectionManagerCallback-1399-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 890505 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@537b6faa, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 890507 DEBUG (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$318/1301159414@dd7eccf
[junit4] 2> 890509 INFO (zkConnectionManagerCallback-1403-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 890757 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 890758 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 890758 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 890758 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 890758 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 890759 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 890762 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 890762 DEBUG (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73802051687809026-127.0.0.1:8983_solr-n_0000000000
[junit4] 2> 890762 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:8983_solr
[junit4] 2> 890763 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.Overseer Overseer (id=73802051687809026-127.0.0.1:8983_solr-n_0000000000) starting
[junit4] 2> 890786 INFO (zkConnectionManagerCallback-1410-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 890787 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 890787 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 890788 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 890788 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 890788 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 890789 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44338/solr ready
[junit4] 2> 890789 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 890792 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 890793 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 890793 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 890803 INFO (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:8983_solr
[junit4] 2> 890803 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 890803 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 890805 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
[junit4] 2> 890808 INFO (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 890809 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
[junit4] 2> 890809 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 890812 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 890814 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 890814 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 890816 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 890816 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 890816 INFO (zkCallback-1409-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 890817 DEBUG (TEST-ZkControllerTest.testPublishAndWaitForDownStates-seed#[C328FE39FAA33E5B]) [n:127.0.0.1:8983_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:8983_solr as DOWN
[junit4] 2> 890829 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 890831 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890832 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 890834 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 890835 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 890840 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 890841 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "node_name":"127.0.0.1:8983_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "name":"testPublishAndWaitForDownStates",
[junit4] 2> "stateFormat":"2"} current state version: 0
[junit4] 2> 890841 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: testPublishAndWaitForDownStates
[junit4] 2> 890841 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890842 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: create/delete: /solr/collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890842 INFO (zkCallback-1402-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/testPublishAndWaitForDownStates/state.json] for collection [testPublishAndWaitForDownStates] has occurred - updating... (live nodes size: [1])
[junit4] 2> 890842 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 890842 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890843 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "node_name":"non_existent_host1",
[junit4] 2> "collection":"testPublishAndWaitForDownStates",
[junit4] 2> "shard":"shard1",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"addreplica"} current state version: 0
[junit4] 2> 890843 INFO (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "node_name":"non_existent_host1",
[junit4] 2> "collection":"testPublishAndWaitForDownStates",
[junit4] 2> "shard":"shard1",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"addreplica"}
[junit4] 2> 890844 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 890844 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node21":{
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "base_url":null,
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"non_existent_host1",
[junit4] 2> "type":null}}}
[junit4] 2> 890844 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "node_name":"non_existent_host2",
[junit4] 2> "collection":"testPublishAndWaitForDownStates",
[junit4] 2> "shard":"shard1",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"addreplica"} current state version: 0
[junit4] 2> 890844 INFO (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "node_name":"non_existent_host2",
[junit4] 2> "collection":"testPublishAndWaitForDownStates",
[junit4] 2> "shard":"shard1",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"addreplica"}
[junit4] 2> 890845 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node21":{
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "base_url":null,
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"non_existent_host1",
[junit4] 2> "type":null}}}
[junit4] 2> 890845 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node21":{
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "base_url":null,
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"non_existent_host1",
[junit4] 2> "type":null},
[junit4] 2> "core_node22":{
[junit4] 2> "core":"testPublishAndWaitForDownStates",
[junit4] 2> "base_url":null,
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"non_existent_host2",
[junit4] 2> "type":null}}}
[junit4] 2> 890845 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:8983_solr"} current state version: 0
[junit4] 2> 890846 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:8983_solr
[junit4] 2> 890846 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 890946 DEBUG (OverseerStateUpdate-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testPublishAndWaitForDownStates/state.json version: 0
[junit4] 2> 890947 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890947 INFO (zkCallback-1402-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testPublishAndWaitForDownStates/state.json] for collection [testPublishAndWaitForDownStates] has occurred - updating... (live nodes size: [1])
[junit4] 2> 890947 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 890947 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/testPublishAndWaitForDownStates/state.json
[junit4] 2> 890949 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 892794 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 893267 INFO (OverseerStateUpdate-73802051630596099-127.0.0.1:8983_solr-n_0000000001) [ ] o.a.s.c.Overseer According to ZK I (id=73802051630596099-127.0.0.1:8983_solr-n_0000000001) am no longer a leader.
[junit4] 2> 893267 INFO (OverseerStateUpdate-73802051630596099-127.0.0.1:8983_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:8983_solr
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 894795 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 896796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 898796 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 898800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 900800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 900800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 900800 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 900801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 900801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 900801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 900801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 902801 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 904802 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 906803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 908803 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 908804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 910804 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 910805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 912805 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 912806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 914806 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 914807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 916807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 916807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 916807 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 916808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 916808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 916808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 916808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 918808 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 920810 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 920811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 922811 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 922812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 924812 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 924813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 926813 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 926814 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 928815 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 928816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 930816 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 930817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 932817 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 932818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 934818 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 936819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 938819 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 938820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 940820 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 940821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 942821 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 942822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 944822 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 944823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 946823 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 946824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 948824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 948824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 948824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 948824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 948824 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 948825 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 948825 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 950874 DEBUG (OverseerCollectionConfigSetProcessor-73802051687809026-127.0.0.1:8983_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 950874 WARN (TEST-Zk
[...truncated too long message...]
owed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null674525468
[copy] Copying 40 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null674525468
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null674525468
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1470687114
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1470687114
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1470687114
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 74ms :: artifacts dl 9ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 238 minutes 45 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)