You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/11/25 14:40:28 UTC
[JENKINS] Lucene-Solr-7.6-Linux (64bit/jdk-9.0.4) - Build # 27 -
Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/27/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC
4 tests failed.
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:38325/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:38325/solr
at __randomizedtesting.SeedInfo.seed([56E0F558F094597D:97108CF4DDC493DA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:40149/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:40149/solr
at __randomizedtesting.SeedInfo.seed([56E0F558F094597D:97108CF4DDC493DA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:46685/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:46685/solr
at __randomizedtesting.SeedInfo.seed([56E0F558F094597D:97108CF4DDC493DA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:42619/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:42619/solr
at __randomizedtesting.SeedInfo.seed([56E0F558F094597D:97108CF4DDC493DA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 13179 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTlogReplica
[junit4] 2> 397000 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/init-core-data-001
[junit4] 2> 397000 WARN (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
[junit4] 2> 397000 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 397001 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 397001 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001
[junit4] 2> 397001 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 397001 INFO (Thread-1428) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 397002 INFO (Thread-1428) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 397007 ERROR (Thread-1428) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 397102 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38823
[junit4] 2> 397104 INFO (zkConnectionManagerCallback-1136-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397108 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
[junit4] 2> 397108 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@561871d4{/solr,null,AVAILABLE}
[junit4] 2> 397109 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ea8124e{/solr,null,AVAILABLE}
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@26f0c849{SSL,[ssl, http/1.1]}{127.0.0.1:34311}
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.e.j.s.Server Started @397214ms
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34311}
[junit4] 2> 397110 ERROR (jetty-launcher-1133-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.6.0
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 397110 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-25T13:51:35.099771Z
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@e4afbe8{SSL,[ssl, http/1.1]}{127.0.0.1:42619}
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.e.j.s.Server Started @397216ms
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42619}
[junit4] 2> 397112 ERROR (jetty-launcher-1133-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.6.0
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 397112 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-25T13:51:35.101419Z
[junit4] 2> 397119 INFO (zkConnectionManagerCallback-1138-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397119 INFO (zkConnectionManagerCallback-1140-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397119 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 397119 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 397122 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10018ee1da00001, likely client has closed socket
[junit4] 2> 397223 INFO (jetty-launcher-1133-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38823/solr
[junit4] 2> 397230 INFO (zkConnectionManagerCallback-1144-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397246 INFO (zkConnectionManagerCallback-1146-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397269 INFO (jetty-launcher-1133-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38823/solr
[junit4] 2> 397270 INFO (zkConnectionManagerCallback-1150-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397276 INFO (zkConnectionManagerCallback-1154-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397281 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42619_solr
[junit4] 2> 397281 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.c.Overseer Overseer (id=72085005016236038-127.0.0.1:42619_solr-n_0000000000) starting
[junit4] 2> 397284 INFO (zkConnectionManagerCallback-1161-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397285 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38823/solr ready
[junit4] 2> 397285 INFO (OverseerStateUpdate-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42619_solr
[junit4] 2> 397285 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42619_solr
[junit4] 2> 397290 INFO (OverseerStateUpdate-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 397291 INFO (zkCallback-1160-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 397294 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 397305 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 397305 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42619.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397306 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 397306 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34311_solr
[junit4] 2> 397307 INFO (zkCallback-1153-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 397307 INFO (zkCallback-1160-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 397307 INFO (zkCallback-1145-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 397311 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42619.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397311 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42619.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397312 INFO (jetty-launcher-1133-thread-1) [n:127.0.0.1:42619_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node1/.
[junit4] 2> 397319 INFO (zkConnectionManagerCallback-1167-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397320 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 397321 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38823/solr ready
[junit4] 2> 397321 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 397329 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34311.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397335 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34311.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397335 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34311.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 397336 INFO (jetty-launcher-1133-thread-2) [n:127.0.0.1:34311_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node2/.
[junit4] 2> 397350 INFO (zkConnectionManagerCallback-1170-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397354 INFO (zkConnectionManagerCallback-1175-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 397355 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 397356 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38823/solr ready
[junit4] 2> 397357 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
[junit4] 2> 397368 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 397368 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 397381 INFO (TEST-TestTlogReplica.testRemoveLeader-seed#[56E0F558F094597D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRemoveLeader
[junit4] 2> 397383 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_remove_leader&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 397384 INFO (OverseerThreadFactory-1866-thread-1-processing-n:127.0.0.1:42619_solr) [n:127.0.0.1:42619_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_remove_leader
[junit4] 2> 397491 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 397503 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 397505 INFO (OverseerStateUpdate-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_remove_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_remove_leader_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:42619/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 397506 INFO (OverseerStateUpdate-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_remove_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_remove_leader_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34311/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 397712 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_remove_leader_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 397712 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_remove_leader_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 397712 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 398719 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
[junit4] 2> 398719 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
[junit4] 2> 398724 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_remove_leader_shard1_replica_t2] Schema name=minimal
[junit4] 2> 398724 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_remove_leader_shard1_replica_t1] Schema name=minimal
[junit4] 2> 398725 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 398725 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 398725 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_remove_leader_shard1_replica_t2' using configuration from collection tlog_replica_test_remove_leader, trusted=true
[junit4] 2> 398725 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_remove_leader_shard1_replica_t1' using configuration from collection tlog_replica_test_remove_leader, trusted=true
[junit4] 2> 398725 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34311.solr.core.tlog_replica_test_remove_leader.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_remove_leader.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 398725 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42619.solr.core.tlog_replica_test_remove_leader.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_remove_leader.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 398725 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_remove_leader_shard1_replica_t2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node2/tlog_replica_test_remove_leader_shard1_replica_t2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node2/./tlog_replica_test_remove_leader_shard1_replica_t2/data/]
[junit4] 2> 398725 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_remove_leader_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node1/tlog_replica_test_remove_leader_shard1_replica_t1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node1/./tlog_replica_test_remove_leader_shard1_replica_t1/data/]
[junit4] 2> 398767 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 398767 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 398768 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 398768 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 398770 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@30920863[tlog_replica_test_remove_leader_shard1_replica_t2] main]
[junit4] 2> 398771 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 398771 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 398772 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 398773 INFO (searcherExecutor-1875-thread-1-processing-n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2 c:tlog_replica_test_remove_leader s:shard1 r:core_node4) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t2] Registered new searcher Searcher@30920863[tlog_replica_test_remove_leader_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 398773 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618114140451110912
[junit4] 2> 398778 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 398778 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 398778 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_remove_leader/terms/shard1 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 398779 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 398779 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 398781 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a8756b2[tlog_replica_test_remove_leader_shard1_replica_t1] main]
[junit4] 2> 398781 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 398781 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 398782 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 398782 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 398782 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618114140460548096
[junit4] 2> 398784 INFO (searcherExecutor-1876-thread-1-processing-n:127.0.0.1:42619_solr x:tlog_replica_test_remove_leader_shard1_replica_t1 c:tlog_replica_test_remove_leader s:shard1 r:core_node3) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t1] Registered new searcher Searcher@a8756b2[tlog_replica_test_remove_leader_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 398787 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_remove_leader/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
[junit4] 2> 399282 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 399282 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 399282 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/
[junit4] 2> 399283 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_remove_leader_shard1_replica_t2 url=https://127.0.0.1:34311/solr START replicas=[https://127.0.0.1:42619/solr/tlog_replica_test_remove_leader_shard1_replica_t1/] nUpdates=100
[junit4] 2> 399283 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_remove_leader_shard1_replica_t2 url=https://127.0.0.1:34311/solr DONE. We have no versions. sync failed.
[junit4] 2> 399288 INFO (qtp1752562000-4804) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 399288 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 399288 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 399288 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 399288 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_remove_leader_shard1_replica_t2 stopping background replication from leader
[junit4] 2> 399291 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/ shard1
[junit4] 2> 399444 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 399481 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_remove_leader_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1769
[junit4] 2> 399546 INFO (zkCallback-1145-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 399791 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_remove_leader_shard1_replica_t1 starting background replication from leader
[junit4] 2> 399791 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 399792 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 399792 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 399795 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_remove_leader_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2083
[junit4] 2> 399796 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 399895 INFO (zkCallback-1145-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 399895 INFO (zkCallback-1153-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 400797 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_remove_leader&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3413
[junit4] 2> 400804 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_remove_leader/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
[junit4] 2> 400820 INFO (qtp1752562000-4722) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/&wt=javabin&version=2}{add=[1 (1618114142578671616)]} 0 11
[junit4] 2> 400820 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1618114142578671616)]} 0 18
[junit4] 2> 400830 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 400834 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1618114142612226048,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 400834 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d76bfc8 commitCommandVersion:1618114142612226048
[junit4] 2> 400838 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4cb445b2[tlog_replica_test_remove_leader_shard1_replica_t2] main]
[junit4] 2> 400838 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 400839 INFO (searcherExecutor-1875-thread-1-processing-n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2 c:tlog_replica_test_remove_leader s:shard1 r:core_node4) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t2] Registered new searcher Searcher@4cb445b2[tlog_replica_test_remove_leader_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.6.0):C1)))}
[junit4] 2> 400840 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 6
[junit4] 2> 400840 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_remove_leader:6&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 19
[junit4] 2> 400844 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 400849 INFO (qtp1752562000-4734) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 400860 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2/
[junit4] 2> 400862 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 400862 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 400862 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 1543153898823
[junit4] 2> 400862 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 400862 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 400862 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 400864 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=0
[junit4] 2> 400864 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Number of files in latest index in master: 18
[junit4] 2> 400865 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@1ec25b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14d3c591)
[junit4] 2> 400871 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2
[junit4] 2> 400879 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400880 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400881 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400882 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.dvdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400883 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400884 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400884 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FST50_0.tfp&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400885 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400886 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FST50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400887 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.mdvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400887 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FST50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400888 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Memory_0.mdvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400889 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400889 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.dvdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400890 INFO (qtp2043848811-4737) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400891 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400892 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 400892 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Bytes downloaded: 2728, Bytes skipped downloading: 0
[junit4] 2> 400892 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=2728) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@1ec25b8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14d3c591)
[junit4] 2> 400898 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 400899 INFO (indexFetcher-1886-thread-1) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@44eb317a[tlog_replica_test_remove_leader_shard1_replica_t1] main]
[junit4] 2> 400900 INFO (searcherExecutor-1876-thread-1) [ ] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t1] Registered new searcher Searcher@44eb317a[tlog_replica_test_remove_leader_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.6.0):C1)))}
[junit4] 2> 400950 INFO (qtp1752562000-4734) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 400955 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node4 x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 400956 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletereplica with params replica=core_node4&action=DELETEREPLICA&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 400957 INFO (OverseerCollectionConfigSetProcessor-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 400958 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_remove_leader.shard1.replica_t2, tag=null
[junit4] 2> 400958 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@36adc5be: rootName = solr_34311, domain = solr.core.tlog_replica_test_remove_leader.shard1.replica_t2, service url = null, agent id = null] for registry solr.core.tlog_replica_test_remove_leader.shard1.replica_t2 / com.codahale.metrics.MetricRegistry@8f3f9da
[junit4] 2> 400973 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_remove_leader_shard1_replica_t2 stopping background replication from leader
[junit4] 2> 400973 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t2] CLOSING SolrCore org.apache.solr.core.SolrCore@72fac4dc
[junit4] 2> 400973 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_remove_leader.shard1.replica_t2, tag=72fac4dc
[junit4] 2> 400973 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tlog_replica_test_remove_leader.shard1.leader, tag=72fac4dc
[junit4] 2> 400977 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_remove_leader/terms/shard1 to Terms{values={core_node3=1}, version=3}
[junit4] 2> 400979 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 400980 INFO (qtp2043848811-4725) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=tlog_replica_test_remove_leader_shard1_replica_t2&deleteMetricsHistory=true&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=21
[junit4] 2> 401080 INFO (zkCallback-1153-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 401081 INFO (zkCallback-1145-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 401083 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replica=core_node4&action=DELETEREPLICA&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2} status=0 QTime=127
[junit4] 2> 402959 INFO (OverseerCollectionConfigSetProcessor-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 403479 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42619/solr/tlog_replica_test_remove_leader_shard1_replica_t1/
[junit4] 2> 403479 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 403480 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SyncStrategy https://127.0.0.1:42619/solr/tlog_replica_test_remove_leader_shard1_replica_t1/ has no replicas
[junit4] 2> 403480 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_remove_leader_shard1_replica_t1 stopping background replication from leader
[junit4] 2> 403480 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext New leader does not have old tlog to replay
[junit4] 2> 403482 INFO (zkCallback-1153-thread-2) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42619/solr/tlog_replica_test_remove_leader_shard1_replica_t1/ shard1
[junit4] 2> 403482 INFO (zkCallback-1153-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 403589 ERROR (TEST-TestTlogReplica.testRemoveLeader-seed#[56E0F558F094597D]) [ ] o.a.s.c.s.i.CloudSolrClient Request to collection [tlog_replica_test_remove_leader] failed due to (404) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34311/solr/tlog_replica_test_remove_leader_shard1_replica_t2: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
[junit4] 2> <title>Error 404 Can not find: /solr/tlog_replica_test_remove_leader_shard1_replica_t2/update</title>
[junit4] 2> </head>
[junit4] 2> <body><h2>HTTP ERROR 404</h2>
[junit4] 2> <p>Problem accessing /solr/tlog_replica_test_remove_leader_shard1_replica_t2/update. Reason:
[junit4] 2> <pre> Can not find: /solr/tlog_replica_test_remove_leader_shard1_replica_t2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>
[junit4] 2>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2> , retry? 0
[junit4] 2> 403593 INFO (qtp1752562000-4734) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1618114145504198656)]} 0 1
[junit4] 2> 403593 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1618114145505247232,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 403594 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3e2feee4 commitCommandVersion:1618114145505247232
[junit4] 2> 403596 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d50cd46[tlog_replica_test_remove_leader_shard1_replica_t1] main]
[junit4] 2> 403596 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 403597 INFO (searcherExecutor-1876-thread-1-processing-n:127.0.0.1:42619_solr x:tlog_replica_test_remove_leader_shard1_replica_t1 c:tlog_replica_test_remove_leader s:shard1 r:core_node3) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t1] Registered new searcher Searcher@3d50cd46[tlog_replica_test_remove_leader_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.6.0):C1) Uninverting(_1(7.6.0):C1)))}
[junit4] 2> 403597 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_remove_leader:8&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 403600 INFO (qtp1752562000-4732) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node3 x:tlog_replica_test_remove_leader_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_remove_leader_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
[junit4] 2> 403601 INFO (qtp2043848811-4733) [n:127.0.0.1:34311_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=tlog_replica_test_remove_leader&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 403604 INFO (qtp2043848811-4735) [n:127.0.0.1:34311_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 403605 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.tlog_replica_test_remove_leader.shard1.replica_t1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 403606 INFO (qtp1752562000-4726) [n:127.0.0.1:42619_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 403608 INFO (qtp1752562000-4736) [n:127.0.0.1:42619_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.tlog_replica_test_remove_leader.shard1.replica_t1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 403608 INFO (OverseerThreadFactory-1866-thread-3-processing-n:127.0.0.1:42619_solr) [n:127.0.0.1:42619_solr c:tlog_replica_test_remove_leader s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34311_solr for creating new replica of shard shard1 for collection tlog_replica_test_remove_leader
[junit4] 2> 403610 INFO (OverseerStateUpdate-72085005016236038-127.0.0.1:42619_solr-n_0000000000) [n:127.0.0.1:42619_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"tlog_replica_test_remove_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_remove_leader_shard1_replica_t5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34311/solr",
[junit4] 2> "node_name":"127.0.0.1:34311_solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 403711 INFO (zkCallback-1153-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_remove_leader/state.json] for collection [tlog_replica_test_remove_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 403811 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&name=tlog_replica_test_remove_leader_shard1_replica_t5&action=CREATE&collection=tlog_replica_test_remove_leader&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 403816 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
[junit4] 2> 403820 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.s.IndexSchema [tlog_replica_test_remove_leader_shard1_replica_t5] Schema name=minimal
[junit4] 2> 403821 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 403821 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_remove_leader_shard1_replica_t5' using configuration from collection tlog_replica_test_remove_leader, trusted=true
[junit4] 2> 403822 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34311.solr.core.tlog_replica_test_remove_leader.shard1.replica_t5' (registry 'solr.core.tlog_replica_test_remove_leader.shard1.replica_t5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@78d6712f
[junit4] 2> 403822 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.c.SolrCore [[tlog_replica_test_remove_leader_shard1_replica_t5] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node2/tlog_replica_test_remove_leader_shard1_replica_t5], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-001/tempDir-001/node2/./tlog_replica_test_remove_leader_shard1_replica_t5/data/]
[junit4] 2> 403855 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 403855 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 403856 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 403856 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 403857 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.s.SolrIndexSearcher Opening [Searcher@21bd4506[tlog_replica_test_remove_leader_shard1_replica_t5] main]
[junit4] 2> 403858 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 403858 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 403859 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 403859 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618114145784168448
[junit4] 2> 403859 INFO (searcherExecutor-1888-thread-1-processing-n:127.0.0.1:34311_solr x:tlog_replica_test_remove_leader_shard1_replica_t5 c:tlog_replica_test_remove_leader s:shard1 r:core_node6) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.c.SolrCore [tlog_replica_test_remove_leader_shard1_replica_t5] Registered new searcher Searcher@21bd4506[tlog_replica_test_remove_leader_shard1_replica_t5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 403861 INFO (qtp2043848811-4799) [n:127.0.0.1:34311_solr c:tlog_replica_test_remove_leader s:shard1 r:core_node6 x:tlog_replica_test_remove_leader_shard1_replica_t5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_remove_leader/term
[...truncated too long message...]
1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 209972 INFO (jetty-closer-134-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3665f1a3{/solr,null,UNAVAILABLE}
[junit4] 2> 209973 INFO (jetty-closer-134-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 209973 INFO (jetty-closer-134-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 209973 INFO (jetty-closer-134-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@774d744c: rootName = solr_37313, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@4f4f48a6
[junit4] 2> 209974 INFO (jetty-closer-134-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37313_solr
[junit4] 2> 209974 INFO (jetty-closer-134-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 209974 INFO (zkCallback-210-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 209974 INFO (zkCallback-203-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 209975 INFO (jetty-closer-134-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72085187952640017-127.0.0.1:37313_solr-n_0000000003) closing
[junit4] 2> 209975 INFO (OverseerStateUpdate-72085187952640017-127.0.0.1:37313_solr-n_0000000003) [n:127.0.0.1:37313_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37313_solr
[junit4] 2> 209975 WARN (OverseerAutoScalingTriggerThread-72085187952640017-127.0.0.1:37313_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 209977 INFO (jetty-closer-134-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@55833be4{/solr,null,UNAVAILABLE}
[junit4] 2> 209977 INFO (jetty-closer-134-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 209980 ERROR (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 209981 INFO (SUITE-TestTlogReplica-seed#[56E0F558F094597D]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34411 34411
[junit4] 2> 210092 INFO (Thread-95) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34411 34411
[junit4] 2> 210093 WARN (Thread-95) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 31 /solr/configs/conf
[junit4] 2> 31 /solr/collections/tlog_replica_test_create_delete/terms/shard1
[junit4] 2> 30 /solr/collections/tlog_replica_test_create_delete/terms/shard2
[junit4] 2> 19 /solr/collections/tlog_replica_test_remove_leader/terms/shard1
[junit4] 2> 18 /solr/collections/tlog_replica_test_recovery/terms/shard1
[junit4] 2> 18 /solr/collections/tlog_replica_test_real_time_get/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_kill_leader/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
[junit4] 2> 15 /solr/aliases.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_delete_by_id/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
[junit4] 2> 8 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
[junit4] 2> 7 /solr/security.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
[junit4] 2> 4 /solr/collections/tlog_replica_test_add_docs/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/collections/tlog_replica_test_create_delete/state.json
[junit4] 2> 15 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
[junit4] 2> 15 /solr/clusterstate.json
[junit4] 2> 15 /solr/clusterprops.json
[junit4] 2> 11 /solr/collections/tlog_replica_test_kill_leader/state.json
[junit4] 2> 9 /solr/collections/tlog_replica_test_remove_leader/state.json
[junit4] 2> 8 /solr/collections/tlog_replica_test_recovery/state.json
[junit4] 2> 8 /solr/collections/tlog_replica_test_basic_leader_election/state.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_real_time_get/state.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/72085187952640017-127.0.0.1:37313_solr-n_0000000003
[junit4] 2> 3 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
[junit4] 2> 3 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
[junit4] 2> 3 /solr/collections/tlog_replica_test_delete_by_id/state.json
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72085187952640006-core_node3-n_0000000002
[junit4] 2> 2 /solr/collections/tlog_replica_test_add_docs/state.json
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72085187952640006-core_node16-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72085187952640006-core_node11-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72085187952640004-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72085187952640004-core_node7-n_0000000001
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72085187952640004-core_node5-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/live_nodes
[junit4] 2> 15 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestTlogReplica_56E0F558F094597D-002
[junit4] 2> Nov 25, 2018 2:39:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {foo=PostingsFormat(name=Direct), title_s=PostingsFormat(name=Asserting), id=FST50, foo_s=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Direct), inplace_updatable_int=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=897, maxMBSortInHeap=7.479584846538516, sim=RandomSimilarity(queryNorm=false): {}, locale=en-NA, timezone=Universal
[junit4] 2> NOTE: Linux 4.15.0-39-generic amd64/Oracle Corporation 9.0.4 (64-bit)/cpus=8,threads=1,free=289241512,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestTlogReplica, TestTlogReplica]
[junit4] Completed [5/5 (5!)] on J2 in 99.49s, 13 tests, 1 failure <<< FAILURES!
[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 65 tests, 5 failures [seed: 56E0F558F094597D]
Total time: 3 minutes 34 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=ad90d8bb44d9470d2f4162642d1aeffb176d6956, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.6-Linux
[WARNINGS] Computing warning deltas based on reference build #26
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-7.6-Linux (64bit/jdk-11) - Build # 32 - Still
Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/32/
Java: 64bit/jdk-11 -XX:-UseCompressedOops -XX:+UseParallelGC
34 tests failed.
FAILED: org.apache.solr.cloud.ReplaceNodeTest.test
Error Message:
Could not find collection : replacenodetest_coll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : replacenodetest_coll
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:805C7F91195FC614]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:87)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.ReplaceNodeTest.test
Error Message:
Could not find collection : replacenodetest_coll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : replacenodetest_coll
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:805C7F91195FC614]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:87)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.ReplaceNodeTest.test
Error Message:
Could not find collection : replacenodetest_coll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : replacenodetest_coll
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:805C7F91195FC614]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:87)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.SplitShardTest.doTest
Error Message:
Error from server at https://127.0.0.1:42381/solr: Could not find collection : splitshardtest-collection
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:42381/solr: Could not find collection : splitshardtest-collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:AF4CF8EFDA18B855]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.SplitShardTest.doTest(SplitShardTest.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.SplitShardTest.doTest
Error Message:
Error from server at https://127.0.0.1:43029/solr: Could not find collection : splitshardtest-collection
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:43029/solr: Could not find collection : splitshardtest-collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:AF4CF8EFDA18B855]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.SplitShardTest.doTest(SplitShardTest.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.SplitShardTest.doTest
Error Message:
Error from server at https://127.0.0.1:45493/solr: SPLITSHARD timed out waiting for subshard leaders to come up
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45493/solr: SPLITSHARD timed out waiting for subshard leaders to come up
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:AF4CF8EFDA18B855]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.SplitShardTest.doTest(SplitShardTest.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.SplitShardTest.doTest
Error Message:
Error from server at https://127.0.0.1:43873/solr: SPLITSHARD timed out waiting for subshard leaders to come up
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:43873/solr: SPLITSHARD timed out waiting for subshard leaders to come up
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:AF4CF8EFDA18B855]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.SplitShardTest.doTest(SplitShardTest.java:64)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:F4B2947F4F831A26]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:157)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth(TestMiniSolrCloudClusterSSL.java:121)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:F4B2947F4F831A26]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:157)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth(TestMiniSolrCloudClusterSSL.java:121)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:5FB905F0775F54FD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName(TestMiniSolrCloudClusterSSL.java:185)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:F4B2947F4F831A26]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:157)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth(TestMiniSolrCloudClusterSSL.java:121)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:5FB905F0775F54FD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName(TestMiniSolrCloudClusterSSL.java:185)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:F4B2947F4F831A26]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:157)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslAndNoClientAuth(TestMiniSolrCloudClusterSSL.java:121)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName
Error Message:
Could not find collection:second_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:second_collection
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:5FB905F0775F54FD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:263)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:249)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithInvalidPeerName(TestMiniSolrCloudClusterSSL.java:185)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:43743/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:43743/solr
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:C9F839E79AF3614B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testCreateDelete {seed=[808404BB7A3ABEC:1318B107415353DC]}
Error Message:
Could not find collection : tlog_replica_test_create_delete
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : tlog_replica_test_create_delete
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:1318B107415353DC]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:258)
at org.apache.solr.cloud.TestTlogReplica.testCreateDelete(TestTlogReplica.java:190)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:34897/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:34897/solr
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:C9F839E79AF3614B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testCreateDelete {seed=[808404BB7A3ABEC:1318B107415353DC]}
Error Message:
Could not find collection : tlog_replica_test_create_delete
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : tlog_replica_test_create_delete
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:1318B107415353DC]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:258)
at org.apache.solr.cloud.TestTlogReplica.testCreateDelete(TestTlogReplica.java:190)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:35881/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:35881/solr
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:C9F839E79AF3614B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:33659/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:33659/solr
at __randomizedtesting.SeedInfo.seed([808404BB7A3ABEC:C9F839E79AF3614B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java
[...truncated too long message...]
rdTest.java:64)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] 2> 689882 DEBUG (ScheduledTrigger-2562-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 690882 DEBUG (ScheduledTrigger-2562-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 691258 WARN (zkCallback-2027-thread-3) [ ] o.a.s.c.c.ZkStateReader$AliasesManager ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /aliases.json]
[junit4] 2> 691259 INFO (jetty-closer-2003-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@4ad05805{SSL,[ssl, http/1.1]}{127.0.0.1:०}
[junit4] 2> 691260 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1626651798
[junit4] 2> 691260 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 691260 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@464d2106: rootName = solr_43029, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@34f75465
[junit4] 2> 691263 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 691263 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1202d726: rootName = solr_43029, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@594eba90
[junit4] 2> 691265 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 691265 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@382461f6: rootName = solr_43029, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@79eaff4d
[junit4] 2> 691266 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43029_solr
[junit4] 2> 691266 DEBUG (OverseerStateUpdate-72092307493486595-127.0.0.1:43029_solr-n_0000000000) [n:127.0.0.1:43029_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:43029_solr"} current state version: 0
[junit4] 2> 691267 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 691267 INFO (zkCallback-2012-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 691267 INFO (zkCallback-2019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 691267 INFO (jetty-closer-2003-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72092307493486595-127.0.0.1:43029_solr-n_0000000000) closing
[junit4] 2> 691267 INFO (OverseerStateUpdate-72092307493486595-127.0.0.1:43029_solr-n_0000000000) [n:127.0.0.1:43029_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43029_solr
[junit4] 2> 691268 WARN (OverseerAutoScalingTriggerThread-72092307493486595-127.0.0.1:43029_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 691268 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 691268 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 691269 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 691269 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 691269 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 691269 DEBUG (jetty-closer-2003-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 691270 INFO (jetty-closer-2003-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@599ba314{/solr,null,UNAVAILABLE}
[junit4] 2> 691270 INFO (jetty-closer-2003-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 691271 ERROR (SUITE-SplitShardTest-seed#[808404BB7A3ABEC]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 691271 INFO (SUITE-SplitShardTest-seed#[808404BB7A3ABEC]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40973 40973
[junit4] 2> 691415 INFO (Thread-1532) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40973 40973
[junit4] 2> 691415 WARN (Thread-1532) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 3 /solr/collections/splitshardtest-collection/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.SplitShardTest_808404BB7A3ABEC-001
[junit4] 2> Nov 26, 2018 8:49:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=381, maxMBSortInHeap=5.25968766055246, sim=RandomSimilarity(queryNorm=false): {}, locale=ne-NP, timezone=Turkey
[junit4] 2> NOTE: Linux 4.15.0-39-generic amd64/Oracle Corporation 11 (64-bit)/cpus=8,threads=1,free=140482960,total=455606272
[junit4] 2> NOTE: All tests run in this JVM: [ReplaceNodeTest, TestMiniSolrCloudClusterSSL, AutoAddReplicasIntegrationTest, SplitShardTest, ReplaceNodeTest, AutoAddReplicasIntegrationTest, TestMiniSolrCloudClusterSSL, TestMiniSolrCloudClusterSSL, TestTlogReplica, TestHdfsCloudBackupRestore, SplitShardTest, SplitShardTest]
[junit4] Completed [30/30 (20!)] on J0 in 36.91s, 2 tests, 1 error <<< FAILURES!
[...truncated 20 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1092: There were test failures: 30 suites, 120 tests, 16 errors, 13 failures [seed: 808404BB7A3ABEC]
Total time: 11 minutes 35 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 2/5 failed: org.apache.solr.cloud.ReplaceNodeTest
[repro] 2/5 failed: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[repro] 3/5 failed: org.apache.solr.cloud.SplitShardTest
[repro] 3/5 failed: org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest
[repro] 5/5 failed: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL
[repro] 5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=ff11881eee313f5754ad39b74caa58d5d7238b2e, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.6-Linux
[WARNINGS] Computing warning deltas based on reference build #31
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-7.6-Linux (64bit/jdk-9.0.4) - Build # 31 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/31/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10003_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10007_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10003_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([359013F8FA02E2C2:B5B076D6EB410A64]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 17
live nodes:[127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10003_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10007_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 40 more
Build Log:
[...truncated 14051 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] 2> 1458378 INFO (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_359013F8FA02E2C2-001/init-core-data-001
[junit4] 2> 1458379 WARN (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=239 numCloses=239
[junit4] 2> 1458379 INFO (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1458379 INFO (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 1458386 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10003_solr
[junit4] 2> 1458386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1458387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1458387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1458387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1458390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10003_solr]
[junit4] 2> 1458390 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 1458390 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 1458390 INFO (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1458394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458398 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1458405 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProviderPerReplicaDetails
[junit4] 2> 1458406 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1458406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1458406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1458406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1458406 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1458406 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2037) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2071) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:844) [?:?]
[junit4] 2> 1458407 WARN (ScheduledTrigger-4535-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger ran but was already closed
[junit4] 2> 1458407 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1458407 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1458407 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1458407 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1458408 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1458408 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1458410 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1458411 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1458411 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1458411 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1458414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr, 127.0.0.1:10007_solr]
[junit4] 2> 1458414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458414 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1458415 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458435 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458455 DEBUG (ScheduledTrigger-4540-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458475 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458498 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458518 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458538 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458559 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458579 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458599 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458611 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection perReplicaDataColl, currentVersion=1
[junit4] 2> 1458612 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
[junit4] 2> 1458612 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
[junit4] 2> 1458619 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458626 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
[junit4] 2> 1458626 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
[junit4] 2> 1458639 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458640 DEBUG (simCloudManagerPool-4539-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
[junit4] 2> 1458641 DEBUG (simCloudManagerPool-4539-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
[junit4] 2> 1458646 DEBUG (simCloudManagerPool-4539-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for perReplicaDataColl / shard1 (currentVersion=4): {"core_node1":{
[junit4] 2> "core":"perReplicaDataColl_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"perReplicaDataColl",
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1458649 DEBUG (simCloudManagerPool-4539-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection perReplicaDataColl, currentVersion=4
[junit4] 2> 1458656 DEBUG (simCloudManagerPool-4539-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
[junit4] 2> 1458656 DEBUG (simCloudManagerPool-4539-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
[junit4] 2> 1458659 DEBUG (ScheduledTrigger-4540-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458667 DEBUG (simCloudManagerPool-4539-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
[junit4] 2> 1458667 DEBUG (simCloudManagerPool-4539-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
[junit4] 2> 1458677 DEBUG (simCloudManagerPool-4539-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
[junit4] 2> 1458677 DEBUG (simCloudManagerPool-4539-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
[junit4] 2> 1458677 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
[junit4] 2> 1458677 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
[junit4] 2> 1458680 DEBUG (ScheduledTrigger-4540-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458688 DEBUG (simCloudManagerPool-4539-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
[junit4] 2> 1458688 DEBUG (simCloudManagerPool-4539-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
[junit4] 2> 1458688 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
[junit4] 2> 1458688 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
[junit4] 2> 1458700 DEBUG (ScheduledTrigger-4540-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458720 DEBUG (ScheduledTrigger-4540-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458740 DEBUG (ScheduledTrigger-4540-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458760 DEBUG (ScheduledTrigger-4540-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458780 DEBUG (ScheduledTrigger-4540-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458801 DEBUG (ScheduledTrigger-4540-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458803 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProviderPerReplicaDetails
[junit4] 2> 1458803 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
[junit4] 2> 1458803 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
[junit4] 2> 1458803 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 5
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETE 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1458805 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddReplica
[junit4] 2> 1458805 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1458805 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1458805 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1458805 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1458805 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458805 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:844) [?:?]
[junit4] 2> 1458806 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1458806 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1458806 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1458806 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1458810 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1458810 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1458814 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1458814 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1458815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
[junit4] 2> 1458815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1458818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr, 127.0.0.1:10007_solr]
[junit4] 2> 1458818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1458818 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458839 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458859 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458879 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458899 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458919 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458940 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458960 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1458980 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459002 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459021 DEBUG (simCloudManagerPool-4542-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
[junit4] 2> 1459021 DEBUG (simCloudManagerPool-4542-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
[junit4] 2> 1459021 DEBUG (simCloudManagerPool-4542-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1459022 DEBUG (simCloudManagerPool-4542-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1459022 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459034 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1459035 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1459035 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1459035 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionAddReplica, currentVersion=12
[junit4] 2> 1459035 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
[junit4] 2> 1459035 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
[junit4] 2> 1459043 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459046 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
[junit4] 2> 1459046 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
[junit4] 2> 1459056 DEBUG (simCloudManagerPool-4542-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionAddReplica, currentVersion=14
[junit4] 2> 1459056 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
[junit4] 2> 1459056 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
[junit4] 2> 1459063 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459067 DEBUG (simCloudManagerPool-4542-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
[junit4] 2> 1459067 DEBUG (simCloudManagerPool-4542-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
[junit4] 2> 1459077 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
[junit4] 2> 1459077 DEBUG (simCloudManagerPool-4542-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionAddReplica / shard1 (currentVersion=16): {"core_node1":{
[junit4] 2> "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testCreateCollectionAddReplica",
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1459077 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
[junit4] 2> 1459083 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459103 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459123 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459143 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459164 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459184 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459204 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459224 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459244 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459264 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459285 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459305 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459325 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459345 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459365 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459386 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459406 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459426 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459446 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459466 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459486 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459507 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459527 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459547 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459567 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459587 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459607 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459628 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459648 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459668 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459688 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459708 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459728 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459749 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459769 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459789 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459809 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459829 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459849 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459869 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459890 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459910 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459930 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459950 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459970 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1459990 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460011 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460031 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460051 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460071 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460091 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460111 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460132 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460152 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460172 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460192 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460212 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460232 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460253 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460273 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460293 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460313 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460333 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460353 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460377 DEBUG (ScheduledTrigger-4543-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460398 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460418 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460438 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460458 DEBUG (ScheduledTrigger-4543-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460478 DEBUG (ScheduledTrigger-4543-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460498 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460518 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460538 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460559 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460579 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460599 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460619 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460639 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460659 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460679 DEBUG (ScheduledTrigger-4543-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460700 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460720 DEBUG (ScheduledTrigger-4543-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460740 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460760 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460780 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460800 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460820 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460840 DEBUG (ScheduledTrigger-4543-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460857 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddReplica
[junit4] 2> 1460857 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
[junit4] 2> 1460858 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
[junit4] 2> 1460858 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 4
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * testCreateCollectionAddReplica
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 1
[junit4] 2> ## activeReplicas 1
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * testCreateCollectionAddReplica 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSimPolicyCloud -Dtests.method=testCreateCollectionAddReplica -Dtests.seed=359013F8FA02E2C2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ee-TG -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 2.06s J1 | TestSimPolicyCloud.testCreateCollectionAddReplica <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for collection to become active
[junit4] > Live Nodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10003_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr]
[junit4] > Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10007_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}
[junit4] > at __randomizedtesting.SeedInfo.seed([359013F8FA02E2C2:B5B076D6EB410A64]:0)
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] > Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 17
[junit4] > live nodes:[127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10003_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr]
[junit4] > collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10007_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/16)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10007_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
[junit4] > ... 40 more
[junit4] 2> 1460861 DEBUG (ScheduledTrigger-4543-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460861 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProvider
[junit4] 2> 1460861 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1460861 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1460862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1460862 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1460862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1460862 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:844) [?:?]
[junit4] 2> 1460862 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1460862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1460862 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1460863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 1460863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1460863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr, 127.0.0.1:10007_solr]
[junit4] 2> 1460863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1460863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1460863 DEBUG (ScheduledTrigger-4546-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460883 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460903 DEBUG (ScheduledTrigger-4546-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460923 DEBUG (ScheduledTrigger-4546-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460944 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460964 DEBUG (ScheduledTrigger-4546-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1460984 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461004 DEBUG (ScheduledTrigger-4546-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461024 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461044 DEBUG (ScheduledTrigger-4546-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461063 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection policiesTest, currentVersion=18
[junit4] 2> 1461063 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
[junit4] 2> 1461063 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
[junit4] 2> 1461064 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461073 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
[junit4] 2> 1461073 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
[junit4] 2> 1461074 DEBUG (simCloudManagerPool-4545-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection policiesTest, currentVersion=20
[junit4] 2> 1461084 DEBUG (simCloudManagerPool-4545-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
[junit4] 2> 1461084 DEBUG (simCloudManagerPool-4545-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
[junit4] 2> 1461084 DEBUG (ScheduledTrigger-4546-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461085 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
[junit4] 2> 1461085 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
[junit4] 2> 1461085 DEBUG (simCloudManagerPool-4545-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / shard1 (currentVersion=21): {"core_node1":{
[junit4] 2> "core":"policiesTest_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1461095 DEBUG (simCloudManagerPool-4545-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
[junit4] 2> 1461095 DEBUG (simCloudManagerPool-4545-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
[junit4] 2> 1461095 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
[junit4] 2> 1461095 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
[junit4] 2> 1461105 DEBUG (ScheduledTrigger-4546-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461118 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProvider
[junit4] 2> 1461118 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
[junit4] 2> 1461118 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
[junit4] 2> 1461119 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * policiesTest
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 2
[junit4] 2> ## activeReplicas 2
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 2
[junit4] 2> ## * policiesTest 2
[junit4] 2> ## - active 2
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - ADDROLE 1
[junit4] 2> ## - CREATE 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1461121 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSplitShard
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1461121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1461121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1461121 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1461121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1461121 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1247) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:844) [?:?]
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1461121 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10003_solr, 127.0.0.1:10007_solr]
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1461122 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1461122 DEBUG (ScheduledTrigger-4549-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461143 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461163 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461183 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461203 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461224 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461244 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461264 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461284 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461304 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461322 DEBUG (simCloudManagerPool-4548-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=25
[junit4] 2> 1461322 DEBUG (simCloudManagerPool-4548-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 25
[junit4] 2> 1461323 DEBUG (simCloudManagerPool-4548-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1461323 DEBUG (simCloudManagerPool-4548-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1461323 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1461323 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1461323 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1461323 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionSplitShard, currentVersion=26
[junit4] 2> 1461323 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=26
[junit4] 2> 1461324 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 26
[junit4] 2> 1461324 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461334 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=27
[junit4] 2> 1461334 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 27
[junit4] 2> 1461335 DEBUG (simCloudManagerPool-4548-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionSplitShard, currentVersion=28
[junit4] 2> 1461344 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461346 DEBUG (simCloudManagerPool-4548-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=28
[junit4] 2> 1461346 DEBUG (simCloudManagerPool-4548-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 28
[junit4] 2> 1461346 DEBUG (simCloudManagerPool-4548-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1 (currentVersion=29): {"core_node1":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1461356 DEBUG (simCloudManagerPool-4548-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=29
[junit4] 2> 1461356 DEBUG (simCloudManagerPool-4548-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 29
[junit4] 2> 1461356 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=30
[junit4] 2> 1461357 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 30
[junit4] 2> 1461365 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461385 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461405 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461425 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461445 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461457 INFO (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.c.SplitShardCmd numSubShards set at: 2
[junit4] 2> 1461465 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461486 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461502 DEBUG (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=31
[junit4] 2> 1461502 DEBUG (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 31
[junit4] 2> 1461502 DEBUG (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** no leader in testCreateCollectionSplitShard / shard1_0:{
[junit4] 2> "parent":"shard1",
[junit4] 2> "stateTimestamp":"1543309140653873000",
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"construction",
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_0_replica_n5",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":20480,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10006/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.9073486328125E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node8":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_0_replica_n7",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":20480,
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "base_url":"http://127.0.0.1:10004/solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.9073486328125E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}}
[junit4] 2> 1461502 DEBUG (simCloudManagerPool-4548-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_1 (currentVersion=32): {"core_node10":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_1_replica_n9",
[junit4] 2> "shard":"shard1_1",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "base_url":"http://127.0.0.1:10004/solr",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1461502 DEBUG (simCloudManagerPool-4548-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_0 (currentVersion=32): {"core_node6":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_0_replica_n5",
[junit4] 2> "shard":"shard1_0",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":20480,
[junit4] 2> "base_url":"http://127.0.0.1:10006/solr",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":1.9073486328125E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1461504 DEBUG (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=32
[junit4] 2> 1461505 DEBUG (simCloudManagerPool-4548-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 32
[junit4] 2> 1461505 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=33
[junit4] 2> 1461505 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 33
[junit4] 2> 1461506 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461526 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461546 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461566 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461586 DEBUG (ScheduledTrigger-4549-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1461606 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSplitShard
[junit4] 2> 1461606 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=34
[junit4] 2> 1461606 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 34
[junit4] 2> 1461606 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 5
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETE 1
[junit4] 2> ## - SPLITSHARD 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1461606 DEBUG (ScheduledTrigger-4549-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] IGNOR/A 0.00s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028"))
[junit4] 2> 1461609 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricsTag
[junit4] 2> 1461609 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1461609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 1461609 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1461609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1461610 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1461610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1461610 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1461610 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 146
[...truncated too long message...]
junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "pullReplicas":"1",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"1",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6}}
[junit4] 2> 1462213 DEBUG (ScheduledTrigger-4555-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1462221 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=49
[junit4] 2> 1462222 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 49
[junit4] 2> 1462222 INFO (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10006_solr for creating new replica of shard s3 for collection policiesTest
[junit4] 2> 1462232 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=50
[junit4] 2> 1462232 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 50
[junit4] 2> 1462232 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=51
[junit4] 2> 1462232 DEBUG (simCloudManagerPool-4554-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 51
[junit4] 2> 1462233 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[359013F8FA02E2C2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] 2> 1462233 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=52
[junit4] 2> 1462233 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 52
[junit4] 2> 1462233 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[359013F8FA02E2C2]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * policiesTest
[junit4] 2> ## shardsTotal 2
[junit4] 2> ## shardsState {active=2}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 6
[junit4] 2> ## activeReplicas 6
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 20,480
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 6
[junit4] 2> ## * policiesTest 6
[junit4] 2> ## - active 6
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - CREATESHARD 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1462233 DEBUG (ScheduledTrigger-4555-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1462234 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1462234 DEBUG (SUITE-TestSimPolicyCloud-seed#[359013F8FA02E2C2]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_359013F8FA02E2C2-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=ee-TG, timezone=Antarctica/Palmer
[junit4] 2> NOTE: Linux 4.15.0-39-generic amd64/Oracle Corporation 9.0.4 (64-bit)/cpus=8,threads=1,free=292650592,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [PrimUtilsTest, TestSolrJ, TestPointFields, ShowFileRequestHandlerTest, TestReplicaProperties, TestLRUCache, TestStressReorder, TestCloudManagedSchema, TestOverriddenPrefixQueryForCustomFieldType, EnumFieldTest, NodeAddedTriggerIntegrationTest, TestPhraseSuggestions, NodeLostTriggerTest, TestIBSimilarityFactory, SolrInfoBeanTest, RankQueryTest, SyncSliceTest, IndexSizeTriggerTest, TestNumericTokenStream, TlogReplayBufferedWhileIndexingTest, SolrSlf4jReporterTest, TestSolrCoreSnapshots, DocExpirationUpdateProcessorFactoryTest, TestAnalyzedSuggestions, ZkFailoverTest, InfixSuggestersTest, TestIntervalFaceting, MaxSizeAutoCommitTest, CollectionStateFormat2Test, TestSimLargeCluster, TestInfoStreamLogging, GraphQueryTest, ShufflingReplicaListTransformerTest, ZkSolrClientTest, XmlUpdateRequestHandlerTest, SharedFSAutoReplicaFailoverTest, TestLockTree, DistributedFacetPivotWhiteBoxTest, UUIDFieldTest, TestTrackingShardHandlerFactory, TestManagedResourceStorage, TestTlogReplica, TaggerTest, TestSolrXml, ReplaceNodeNoTargetTest, TestMultiValuedNumericRangeQuery, SchemaVersionSpecificBehaviorTest, FileUtilsTest, TestRequestForwarding, ReplicaListTransformerTest, AutoAddReplicasPlanActionTest, TestPHPSerializedResponseWriter, TestSlowCompositeReaderWrapper, TestCloudJSONFacetSKG, TestNumericTerms32, SortSpecParsingTest, TestCloudDeleteByQuery, TestNonDefinedSimilarityFactory, InfoHandlerTest, MetricsHistoryIntegrationTest, UpdateLogTest, CdcrWithNodesRestartsTest, DocumentAnalysisRequestHandlerTest, IndexSchemaTest, SuggesterTest, TestFieldSortValues, TestRawTransformer, TestChildDocTransformer, TestUniqueKeyFieldResource, MinimalSchemaTest, TestCollectionsAPIViaSolrCloudCluster, IgnoreLargeDocumentProcessorFactoryTest, TestMacros, ResponseBuilderTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestBinaryResponseWriter, TestFoldingMultitermQuery, DistributedExpandComponentTest, TestCSVLoader, MergeStrategyTest, NoCacheHeaderTest, TestHighlightDedupGrouping, HdfsRestartWhileUpdatingTest, TestScoreJoinQPNoScore, TestFreeTextSuggestions, SolrCloudExampleTest, TestSchemaSimilarityResource, DeleteStatusTest, WordBreakSolrSpellCheckerTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, BasicDistributedZkTest, ShardSplitTest, TestSimPolicyCloud]
[junit4] Completed [435/836 (1!)] on J1 in 3.87s, 7 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 43912 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/31/consoleText
[repro] Revision: ad90d8bb44d9470d2f4162642d1aeffb176d6956
[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseG1GC"
[repro] Repro line: ant test -Dtestcase=TestSimPolicyCloud -Dtests.method=testCreateCollectionAddReplica -Dtests.seed=359013F8FA02E2C2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ee-TG -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[repro] ant clean
[...truncated 6 lines...]
[repro] Test suites by module:
[repro] solr/core
[repro] TestSimPolicyCloud
[repro] ant compile-test
[...truncated 3585 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSimPolicyCloud" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseG1GC" -Dtests.seed=359013F8FA02E2C2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ee-TG -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[...truncated 106 lines...]
[repro] Failures:
[repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[repro] Exiting with code 0
[...truncated 40 lines...]
[JENKINS] Lucene-Solr-7.6-Linux (64bit/jdk1.8.0_172) - Build # 30 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/30/
Java: 64bit/jdk1.8.0_172 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([F0D22745623A5186:70F2426B7379B920]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 18
live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 41 more
Build Log:
[...truncated 14466 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_F0D22745623A5186-001/init-core-data-001
[junit4] 2> 1942183 WARN (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=35 numCloses=35
[junit4] 2> 1942184 INFO (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1942185 INFO (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1942186 INFO (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1942209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1942209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1942209 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1942210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1942210 DEBUG (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 1942210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 1942210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1942210 DEBUG (ScheduledTrigger-10241-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1942210 DEBUG (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 1942212 DEBUG (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 1942212 INFO (SUITE-TestSimPolicyCloud-seed#[F0D22745623A5186]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] IGNOR/A 0.00s J0 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 1942216 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] 2> 1942216 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1942216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1942217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1942217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1942218 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1942218 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1942219 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1942219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1942219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1942220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1942220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1942220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1942220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1942220 DEBUG (ScheduledTrigger-10246-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942241 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942261 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942281 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942301 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942322 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942342 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942362 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942382 DEBUG (ScheduledTrigger-10246-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942402 DEBUG (ScheduledTrigger-10246-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942420 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.TestSimPolicyCloud NRT 127.0.0.1:10001_solr PULL 127.0.0.1:10004_solr , TLOG 127.0.0.1:10000_solr
[junit4] 2> 1942422 DEBUG (simCloudManagerPool-10245-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
[junit4] 2> 1942422 DEBUG (simCloudManagerPool-10245-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
[junit4] 2> 1942422 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942423 DEBUG (simCloudManagerPool-10245-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1942423 DEBUG (simCloudManagerPool-10245-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1942424 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1942424 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942424 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1942425 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection policiesTest, currentVersion=2
[junit4] 2> 1942426 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
[junit4] 2> 1942426 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
[junit4] 2> 1942438 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
[junit4] 2> 1942441 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
[junit4] 2> 1942442 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942448 DEBUG (simCloudManagerPool-10245-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection policiesTest, currentVersion=4
[junit4] 2> 1942459 DEBUG (simCloudManagerPool-10245-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
[junit4] 2> 1942460 DEBUG (simCloudManagerPool-10245-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
[junit4] 2> 1942462 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942466 DEBUG (simCloudManagerPool-10245-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / s1 (currentVersion=5): {"core_node1":{
[junit4] 2> "core":"policiesTest_s1_replica_n1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1942474 DEBUG (simCloudManagerPool-10245-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
[junit4] 2> 1942475 DEBUG (simCloudManagerPool-10245-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
[junit4] 2> 1942483 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942485 DEBUG (simCloudManagerPool-10245-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
[junit4] 2> 1942486 DEBUG (simCloudManagerPool-10245-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
[junit4] 2> 1942486 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
[junit4] 2> 1942486 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
[junit4] 2> 1942503 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942523 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942543 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942563 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942583 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942592 INFO (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica of shard s3 for collection policiesTest
[junit4] 2> 1942603 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
[junit4] 2> 1942603 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
[junit4] 2> 1942603 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942604 INFO (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica of shard s3 for collection policiesTest
[junit4] 2> 1942604 DEBUG (simCloudManagerPool-10245-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / s3 (currentVersion=9): {"core_node5":{
[junit4] 2> "core":"policiesTest_s3_replica_n4",
[junit4] 2> "shard":"s3",
[junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "pullReplicas":"1",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"1",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6}}
[junit4] 2> 1942614 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
[junit4] 2> 1942615 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
[junit4] 2> 1942615 INFO (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10004_solr for creating new replica of shard s3 for collection policiesTest
[junit4] 2> 1942624 DEBUG (ScheduledTrigger-10246-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942625 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
[junit4] 2> 1942625 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
[junit4] 2> 1942626 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
[junit4] 2> 1942626 DEBUG (simCloudManagerPool-10245-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
[junit4] 2> 1942626 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] 2> 1942627 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
[junit4] 2> 1942628 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
[junit4] 2> 1942644 DEBUG (ScheduledTrigger-10246-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942656 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * policiesTest
[junit4] 2> ## shardsTotal 2
[junit4] 2> ## shardsState {active=2}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 6
[junit4] 2> ## activeReplicas 6
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 20,480
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 6
[junit4] 2> ## * policiesTest 6
[junit4] 2> ## - active 6
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - CREATESHARD 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1942662 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddReplica
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1942662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1942662 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1942662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1942662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1942662 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1942662 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1942663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1942663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1942663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion -1
[junit4] 2> 1942664 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1942664 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1942664 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942664 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1942664 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942685 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942705 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942725 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942745 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942765 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942786 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942806 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942826 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942846 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942865 DEBUG (simCloudManagerPool-10248-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
[junit4] 2> 1942865 DEBUG (simCloudManagerPool-10248-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
[junit4] 2> 1942866 DEBUG (simCloudManagerPool-10248-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1942866 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942866 DEBUG (simCloudManagerPool-10248-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1942867 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1942867 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1942868 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1942868 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionAddReplica, currentVersion=14
[junit4] 2> 1942868 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
[junit4] 2> 1942868 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
[junit4] 2> 1942879 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
[junit4] 2> 1942879 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
[junit4] 2> 1942880 DEBUG (simCloudManagerPool-10248-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionAddReplica, currentVersion=16
[junit4] 2> 1942886 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942890 DEBUG (simCloudManagerPool-10248-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
[junit4] 2> 1942890 DEBUG (simCloudManagerPool-10248-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
[junit4] 2> 1942890 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
[junit4] 2> 1942891 DEBUG (simCloudManagerPool-10248-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionAddReplica / shard1 (currentVersion=17): {"core_node1":{
[junit4] 2> "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testCreateCollectionAddReplica",
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1942891 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
[junit4] 2> 1942907 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942927 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942947 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942967 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1942987 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943007 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943028 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943048 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943068 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943088 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943108 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943128 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943149 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943169 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943189 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943209 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943229 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943249 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943270 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943290 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943310 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943330 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943350 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943370 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943391 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943411 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943431 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943451 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943471 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943491 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943512 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943532 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943552 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943572 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943592 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943613 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943633 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943653 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943673 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943693 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943713 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943734 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943754 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943774 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943794 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943814 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943834 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943855 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943875 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943895 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943915 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943935 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943956 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943976 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1943996 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944016 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944036 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944057 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944077 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944097 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944117 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944137 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944157 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944178 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944198 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944218 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944238 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944258 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944278 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944299 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944319 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944339 DEBUG (ScheduledTrigger-10249-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944359 DEBUG (ScheduledTrigger-10249-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944379 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944400 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944420 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944440 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944460 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944480 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944501 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944521 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944541 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944561 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944581 DEBUG (ScheduledTrigger-10249-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944602 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944622 DEBUG (ScheduledTrigger-10249-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944642 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944662 DEBUG (ScheduledTrigger-10249-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944681 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddReplica
[junit4] 2> 1944681 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
[junit4] 2> 1944681 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
[junit4] 2> 1944681 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 4
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * testCreateCollectionAddReplica
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 1
[junit4] 2> ## activeReplicas 1
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * testCreateCollectionAddReplica 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1944682 DEBUG (ScheduledTrigger-10249-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSimPolicyCloud -Dtests.method=testCreateCollectionAddReplica -Dtests.seed=F0D22745623A5186 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-GT -Dtests.timezone=Africa/Bujumbura -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 2.03s J0 | TestSimPolicyCloud.testCreateCollectionAddReplica <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for collection to become active
[junit4] > Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] > Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10002_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}
[junit4] > at __randomizedtesting.SeedInfo.seed([F0D22745623A5186:70F2426B7379B920]:0)
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 18
[junit4] > live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] > collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10002_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/17)={
[junit4] > "replicationFactor":"1",
[junit4] > "pullReplicas":"0",
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"1",
[junit4] > "tlogReplicas":"0",
[junit4] > "autoCreated":"true",
[junit4] > "policy":"c1",
[junit4] > "shards":{"shard1":{
[junit4] > "replicas":{"core_node1":{
[junit4] > "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] > "SEARCHER.searcher.maxDoc":0,
[junit4] > "SEARCHER.searcher.deletedDocs":0,
[junit4] > "INDEX.sizeInBytes":10240,
[junit4] > "node_name":"127.0.0.1:10002_solr",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] > "SEARCHER.searcher.numDocs":0}},
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active"}}}
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
[junit4] > at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
[junit4] > ... 41 more
[junit4] 2> 1944688 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProvider
[junit4] 2> 1944688 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1944688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1944688 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1944688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1944688 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1944688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1944689 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1944689 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1944689 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1944689 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1944689 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1944689 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1944689 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1944689 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1944690 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion -1
[junit4] 2> 1944690 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1944690 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1944690 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1944690 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1944690 DEBUG (ScheduledTrigger-10252-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944711 DEBUG (ScheduledTrigger-10252-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944731 DEBUG (ScheduledTrigger-10252-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944751 DEBUG (ScheduledTrigger-10252-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944771 DEBUG (ScheduledTrigger-10252-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944791 DEBUG (ScheduledTrigger-10252-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944812 DEBUG (ScheduledTrigger-10252-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944832 DEBUG (ScheduledTrigger-10252-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944852 DEBUG (ScheduledTrigger-10252-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944872 DEBUG (ScheduledTrigger-10252-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944889 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection policiesTest, currentVersion=19
[junit4] 2> 1944890 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
[junit4] 2> 1944890 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
[junit4] 2> 1944892 DEBUG (ScheduledTrigger-10252-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944900 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
[junit4] 2> 1944900 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
[junit4] 2> 1944901 DEBUG (simCloudManagerPool-10251-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection policiesTest, currentVersion=21
[junit4] 2> 1944911 DEBUG (simCloudManagerPool-10251-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
[junit4] 2> 1944911 DEBUG (simCloudManagerPool-10251-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
[junit4] 2> 1944912 DEBUG (simCloudManagerPool-10251-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / shard1 (currentVersion=22): {"core_node10":{
[junit4] 2> "core":"policiesTest_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1944912 DEBUG (ScheduledTrigger-10252-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944921 DEBUG (simCloudManagerPool-10251-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
[junit4] 2> 1944922 DEBUG (simCloudManagerPool-10251-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
[junit4] 2> 1944922 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
[junit4] 2> 1944922 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
[junit4] 2> 1944932 DEBUG (ScheduledTrigger-10252-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944945 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProvider
[junit4] 2> 1944945 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
[junit4] 2> 1944945 DEBUG (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
[junit4] 2> 1944945 INFO (TEST-TestSimPolicyCloud.testDataProvider-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * policiesTest
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 2
[junit4] 2> ## activeReplicas 2
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 2
[junit4] 2> ## * policiesTest 2
[junit4] 2> ## - active 2
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - ADDROLE 1
[junit4] 2> ## - CREATE 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1944950 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMetricsTag
[junit4] 2> 1944950 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1944950 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1944950 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1944950 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1944950 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1944950 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1944950 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1944950 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1944950 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1944950 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1944951 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1944951 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion -1
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1944951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1944951 DEBUG (ScheduledTrigger-10255-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944972 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1944992 DEBUG (ScheduledTrigger-10255-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945013 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945033 DEBUG (ScheduledTrigger-10255-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945053 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945073 DEBUG (ScheduledTrigger-10255-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945093 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945113 DEBUG (ScheduledTrigger-10255-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945133 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945152 ERROR (simCloudManagerPool-10254-thread-1) [ ] o.a.s.h.RequestHandlerBase org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{set-cluster-policy=[{cores=<10, node=#ANY}, {replica=<2, shard=#EACH, node=#ANY}, {metrics:abc=overseer, replica=0}], errorMessages=[Invalid metrics: param in {
[junit4] 2> "metrics:abc":"overseer",
[junit4] 2> "replica":0} must have at 2 or 3 segments after 'metrics:' separated by ':']}],
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.processOps(AutoScalingHandler.java:210)
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleRequestBody(AutoScalingHandler.java:148)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:706)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:651)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1945152 DEBUG (simCloudManagerPool-10254-thread-1) [ ] o.a.s.c.a.s.SimCloudManager -- exception handling request
[junit4] 2> org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.processOps(AutoScalingHandler.java:210) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleRequestBody(AutoScalingHandler.java:148) ~[java/:?]
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:706) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:651) ~[test/:?]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 1945153 DEBUG (simCloudManagerPool-10254-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=25
[junit4] 2> 1945153 DEBUG (simCloudManagerPool-10254-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 25
[junit4] 2> 1945153 DEBUG (simCloudManagerPool-10254-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1945153 DEBUG (simCloudManagerPool-10254-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1945154 DEBUG (ScheduledTrigger-10255-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945154 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 1945154 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1945154 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 1945154 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection metricsTest, currentVersion=26
[junit4] 2> 1945154 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=26
[junit4] 2> 1945154 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 26
[junit4] 2> 1945164 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=27
[junit4] 2> 1945165 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 27
[junit4] 2> 1945166 DEBUG (simCloudManagerPool-10254-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection metricsTest, currentVersion=28
[junit4] 2> 1945166 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=28
[junit4] 2> 1945166 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 28
[junit4] 2> 1945174 DEBUG (ScheduledTrigger-10255-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 1945176 DEBUG (simCloudManagerPool-10254-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=29
[junit4] 2> 1945176 DEBUG (simCloudManagerPool-10254-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 29
[junit4] 2> 1945177 DEBUG (simCloudManagerPool-10254-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for metricsTest / shard1 (currentVersion=30): {"core_node1":{
[junit4] 2> "core":"metricsTest_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"metricsTest",
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 1945178 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMetricsTag
[junit4] 2> 1945178 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=30
[junit4] 2> 1945178 DEBUG (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 30
[junit4] 2> 1945178 INFO (TEST-TestSimPolicyCloud.testMetricsTag-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 4
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * metricsTest
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 1
[junit4] 2> ## activeReplicas 1
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 1
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 1
[junit4] 2> ## * metricsTest 1
[junit4] 2> ## - active 1
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 1945182 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[F0D22745623A5186]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProviderPerReplicaDetails
[junit4] 2> 1945182 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 1945182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 1945182 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[F0D22745623A5186]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1945182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1945182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[...truncated too long message...]
kspace/Lucene-Solr-7.6-Linux/solr/core/build.xml
-clover.disable:
ivy-configure:
[ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/top-level-ivy-settings.xml
install-junit4-taskdef:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.codehaus.groovy#groovy-all;2.4.15 in public
[ivy:cachepath] :: resolution report :: resolve 18ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 1 | 0 | 0 | 0 || 1 | 0 |
---------------------------------------------------------------------
-init-totals:
-test:
[mkdir] Created dir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test
[junit4:pickseed] Seed property 'tests.seed' already defined: F0D22745623A5186
[mkdir] Created dir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/temp
[junit4] <JUnit4> says g'day! Master seed: F0D22745623A5186
[junit4] Executing 5 suites with 3 JVMs.
[junit4]
[junit4] Started J0 PID(31776@serv1.sd-datasolutions.de).
[junit4] Started J2 PID(31784@serv1.sd-datasolutions.de).
[junit4] Started J1 PID(31817@serv1.sd-datasolutions.de).
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] OK 0.76s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] OK 0.46s J1 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] OK 0.27s J1 | TestSimPolicyCloud.testCreateCollectionAddReplica
[junit4] OK 0.28s J1 | TestSimPolicyCloud.testDataProvider
[junit4] OK 0.24s J1 | TestSimPolicyCloud.testMetricsTag
[junit4] OK 0.38s J1 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
[junit4] OK 0.52s J1 | TestSimPolicyCloud.testCreateCollectionSplitShard
[junit4] Completed [1/5] on J1 in 5.74s, 7 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] OK 0.84s J2 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] OK 0.45s J2 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] OK 0.26s J2 | TestSimPolicyCloud.testCreateCollectionAddReplica
[junit4] OK 0.28s J2 | TestSimPolicyCloud.testDataProvider
[junit4] OK 0.24s J2 | TestSimPolicyCloud.testMetricsTag
[junit4] OK 0.38s J2 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
[junit4] OK 0.53s J2 | TestSimPolicyCloud.testCreateCollectionSplitShard
[junit4] Completed [2/5] on J2 in 5.83s, 7 tests
[junit4]
[junit4] Duplicate suite name used with XML reports: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] OK 0.83s J0 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] OK 0.43s J0 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] OK 0.27s J0 | TestSimPolicyCloud.testCreateCollectionAddReplica
[junit4] OK 0.27s J0 | TestSimPolicyCloud.testDataProvider
[junit4] OK 0.25s J0 | TestSimPolicyCloud.testMetricsTag
[junit4] OK 0.39s J0 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
[junit4] OK 0.52s J0 | TestSimPolicyCloud.testCreateCollectionSplitShard
[junit4] Completed [3/5] on J0 in 5.96s, 7 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] OK 0.38s J1 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] OK 0.41s J1 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] OK 0.25s J1 | TestSimPolicyCloud.testCreateCollectionAddReplica
[junit4] OK 0.26s J1 | TestSimPolicyCloud.testDataProvider
[junit4] OK 0.23s J1 | TestSimPolicyCloud.testMetricsTag
[junit4] OK 0.37s J1 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
[junit4] OK 0.49s J1 | TestSimPolicyCloud.testCreateCollectionSplitShard
[junit4] Completed [4/5] on J1 in 2.44s, 7 tests
[junit4]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] OK 0.37s J2 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] OK 0.40s J2 | TestSimPolicyCloud.testCreateCollectionAddShardWithReplicaTypeUsingPolicy
[junit4] OK 0.25s J2 | TestSimPolicyCloud.testCreateCollectionAddReplica
[junit4] OK 0.26s J2 | TestSimPolicyCloud.testDataProvider
[junit4] OK 0.23s J2 | TestSimPolicyCloud.testMetricsTag
[junit4] OK 0.38s J2 | TestSimPolicyCloud.testDataProviderPerReplicaDetails
[junit4] OK 0.50s J2 | TestSimPolicyCloud.testCreateCollectionSplitShard
[junit4] Completed [5/5] on J2 in 2.46s, 7 tests
[junit4]
[junit4] JVM J0: 0.93 .. 8.02 = 7.09s
[junit4] JVM J1: 0.96 .. 10.42 = 9.47s
[junit4] JVM J2: 0.93 .. 10.53 = 9.60s
[junit4] Execution time total: 10 seconds
[junit4] Tests summary: 5 suites, 35 tests
[junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J0 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J0/temp])
[junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp])
[junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J2 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J2/temp])
[echo] 5 slowest tests:
[junit4:tophints] 699.68s | org.apache.solr.cloud.cdcr.CdcrOpsAndBoundariesTest
[junit4:tophints] 478.53s | org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4:tophints] 367.88s | org.apache.solr.cloud.cdcr.CdcrWithNodesRestartsTest
[junit4:tophints] 150.06s | org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4:tophints] 139.80s | org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest
-check-totals:
test-nocompile:
BUILD SUCCESSFUL
Total time: 12 seconds
[repro] Failures:
[repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[repro] Exiting with code 0
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2845)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2724)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2705)
at hudson.FilePath.act(FilePath.java:1076)
at hudson.FilePath.act(FilePath.java:1059)
at hudson.FilePath.validateAntFileMask(FilePath.java:2703)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=ad90d8bb44d9470d2f4162642d1aeffb176d6956, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.6-Linux
[WARNINGS] Computing warning deltas based on reference build #29
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-7.6-Linux (64bit/jdk-9.0.4) - Build # 29 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/29/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseG1GC
4 tests failed.
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:44127/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:44127/solr
at __randomizedtesting.SeedInfo.seed([C1E37A5B14169792:1303F739465D35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:43077/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:43077/solr
at __randomizedtesting.SeedInfo.seed([C1E37A5B14169792:1303F739465D35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:34757/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:34757/solr
at __randomizedtesting.SeedInfo.seed([C1E37A5B14169792:1303F739465D35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 7 in https://127.0.0.1:39937/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:39937/solr
at __randomizedtesting.SeedInfo.seed([C1E37A5B14169792:1303F739465D35]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12964 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTlogReplica
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/init-core-data-001
[junit4] 2> 446798 WARN (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
[junit4] 2> 446798 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 446799 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 446799 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 446800 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001
[junit4] 2> 446800 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 446800 INFO (Thread-589) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 446800 INFO (Thread-589) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 446801 ERROR (Thread-589) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 446900 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:33241
[junit4] 2> 446909 INFO (zkConnectionManagerCallback-761-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446912 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
[junit4] 2> 446912 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
[junit4] 2> 446913 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 446913 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 446913 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 446913 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b55ac45{/solr,null,AVAILABLE}
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1559854d{SSL,[ssl, http/1.1]}{127.0.0.1:39227}
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.e.j.s.Server Started @447034ms
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39227}
[junit4] 2> 446915 ERROR (jetty-launcher-758-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.6.0
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 446915 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-26T00:53:53.671944Z
[junit4] 2> 446918 INFO (zkConnectionManagerCallback-763-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446918 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 446918 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 446918 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 446918 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 446918 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@518f85f3{/solr,null,AVAILABLE}
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@20003c7e{SSL,[ssl, http/1.1]}{127.0.0.1:39937}
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.e.j.s.Server Started @447038ms
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39937}
[junit4] 2> 446919 ERROR (jetty-launcher-758-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.6.0
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 446919 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-26T00:53:53.675869Z
[junit4] 2> 446924 INFO (zkConnectionManagerCallback-765-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 446924 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 446994 INFO (jetty-launcher-758-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33241/solr
[junit4] 2> 446998 INFO (zkConnectionManagerCallback-769-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447000 INFO (zkConnectionManagerCallback-771-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447045 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39227_solr
[junit4] 2> 447046 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.c.Overseer Overseer (id=72087609322831876-127.0.0.1:39227_solr-n_0000000000) starting
[junit4] 2> 447049 INFO (zkConnectionManagerCallback-778-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447051 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33241/solr ready
[junit4] 2> 447052 INFO (OverseerStateUpdate-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:39227_solr
[junit4] 2> 447059 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39227_solr
[junit4] 2> 447067 INFO (zkCallback-770-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 447068 INFO (zkCallback-777-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 447068 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 447081 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39227.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447086 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39227.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447086 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39227.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447087 INFO (jetty-launcher-758-thread-1) [n:127.0.0.1:39227_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node1/.
[junit4] 2> 447100 INFO (jetty-launcher-758-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33241/solr
[junit4] 2> 447119 INFO (zkConnectionManagerCallback-783-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447122 INFO (zkConnectionManagerCallback-785-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447125 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 447127 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 447127 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39937_solr
[junit4] 2> 447130 INFO (zkCallback-770-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 447130 INFO (zkCallback-777-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 447139 INFO (zkCallback-784-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 447161 INFO (zkConnectionManagerCallback-792-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447172 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 447172 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33241/solr ready
[junit4] 2> 447176 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 447189 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39937.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447194 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39937.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447194 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39937.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 447195 INFO (jetty-launcher-758-thread-2) [n:127.0.0.1:39937_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node2/.
[junit4] 2> 447227 INFO (zkConnectionManagerCallback-795-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447236 INFO (zkConnectionManagerCallback-800-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 447237 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 447237 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33241/solr ready
[junit4] 2> 447239 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
[junit4] 2> 447254 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447255 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 447262 INFO (TEST-TestTlogReplica.testDeleteById-seed#[C1E37A5B14169792]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteById
[junit4] 2> 447264 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_delete_by_id&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 447265 INFO (OverseerThreadFactory-1496-thread-1-processing-n:127.0.0.1:39227_solr) [n:127.0.0.1:39227_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_delete_by_id
[junit4] 2> 447373 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 447385 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 447387 INFO (OverseerStateUpdate-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_delete_by_id",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_delete_by_id_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39937/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 447388 INFO (OverseerStateUpdate-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_delete_by_id",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_delete_by_id_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39227/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 447593 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_delete_by_id_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_delete_by_id&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 447593 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_delete_by_id_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_delete_by_id&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 447593 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 448600 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
[junit4] 2> 448600 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
[junit4] 2> 448605 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_delete_by_id_shard1_replica_t1] Schema name=minimal
[junit4] 2> 448605 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_delete_by_id_shard1_replica_t2] Schema name=minimal
[junit4] 2> 448612 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 448612 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 448612 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_delete_by_id_shard1_replica_t1' using configuration from collection tlog_replica_test_delete_by_id, trusted=true
[junit4] 2> 448612 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_delete_by_id_shard1_replica_t2' using configuration from collection tlog_replica_test_delete_by_id, trusted=true
[junit4] 2> 448613 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39937.solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 448613 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39227.solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43c638a0
[junit4] 2> 448613 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_delete_by_id_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node2/tlog_replica_test_delete_by_id_shard1_replica_t1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node2/./tlog_replica_test_delete_by_id_shard1_replica_t1/data/]
[junit4] 2> 448613 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_delete_by_id_shard1_replica_t2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node1/tlog_replica_test_delete_by_id_shard1_replica_t2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-001/tempDir-001/node1/./tlog_replica_test_delete_by_id_shard1_replica_t2/data/]
[junit4] 2> 448647 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 448647 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 448648 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 448648 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 448649 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@e3ce0ca[tlog_replica_test_delete_by_id_shard1_replica_t2] main]
[junit4] 2> 448650 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 448650 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 448650 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 448650 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 448651 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 448651 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618155809440530432
[junit4] 2> 448651 INFO (searcherExecutor-1509-thread-1-processing-n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2 c:tlog_replica_test_delete_by_id s:shard1 r:core_node4) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t2] Registered new searcher Searcher@e3ce0ca[tlog_replica_test_delete_by_id_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 448652 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 448652 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 448653 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@73ae9078[tlog_replica_test_delete_by_id_shard1_replica_t1] main]
[junit4] 2> 448654 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 448654 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_delete_by_id/terms/shard1 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 448655 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 448656 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 448656 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 448656 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618155809445773312
[junit4] 2> 448656 INFO (searcherExecutor-1508-thread-1-processing-n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1 c:tlog_replica_test_delete_by_id s:shard1 r:core_node3) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t1] Registered new searcher Searcher@73ae9078[tlog_replica_test_delete_by_id_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 448659 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_delete_by_id/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
[junit4] 2> 449157 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 449157 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 449157 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/
[junit4] 2> 449158 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_delete_by_id_shard1_replica_t2 url=https://127.0.0.1:39227/solr START replicas=[https://127.0.0.1:39937/solr/tlog_replica_test_delete_by_id_shard1_replica_t1/] nUpdates=100
[junit4] 2> 449158 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_delete_by_id_shard1_replica_t2 url=https://127.0.0.1:39227/solr DONE. We have no versions. sync failed.
[junit4] 2> 449159 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 449160 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 449160 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 449160 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 449160 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_delete_by_id_shard1_replica_t2 stopping background replication from leader
[junit4] 2> 449163 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/ shard1
[junit4] 2> 449315 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 449317 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_delete_by_id_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_delete_by_id&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1723
[junit4] 2> 449417 INFO (zkCallback-770-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_delete_by_id/state.json] for collection [tlog_replica_test_delete_by_id] has occurred - updating... (live nodes size: [2])
[junit4] 2> 449660 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_delete_by_id_shard1_replica_t1 starting background replication from leader
[junit4] 2> 449661 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 449662 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 449662 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 449663 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_delete_by_id_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_delete_by_id&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2070
[junit4] 2> 449665 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 449763 INFO (zkCallback-770-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_delete_by_id/state.json] for collection [tlog_replica_test_delete_by_id] has occurred - updating... (live nodes size: [2])
[junit4] 2> 449764 INFO (zkCallback-784-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_delete_by_id/state.json] for collection [tlog_replica_test_delete_by_id] has occurred - updating... (live nodes size: [2])
[junit4] 2> 450665 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_delete_by_id&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3401
[junit4] 2> 450673 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1618155811555508224&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&wt=javabin&version=2}{deleteByQuery=*:* (-1618155811555508224)} 0 0
[junit4] 2> 450675 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 450677 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1618155811564945408,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 450677 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 450677 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 450677 INFO (qtp1675747423-3001) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 450677 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_delete_by_id/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
[junit4] 2> 450678 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_delete_by_id:6&commit=true&wt=javabin&version=2}{deleteByQuery=*:* (-1618155811555508224),commit=} 0 9
[junit4] 2> 450681 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&wt=javabin&version=2}{add=[1 (1618155811567042560)]} 0 0
[junit4] 2> 450681 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_delete_by_id:6&wt=javabin&version=2}{add=[1 (1618155811567042560)]} 0 2
[junit4] 2> 450683 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1618155811571236864,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 450683 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 450683 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@221bf366 commitCommandVersion:1618155811571236864
[junit4] 2> 450686 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5342bca4[tlog_replica_test_delete_by_id_shard1_replica_t2] main]
[junit4] 2> 450687 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 450687 INFO (searcherExecutor-1509-thread-1-processing-n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2 c:tlog_replica_test_delete_by_id s:shard1 r:core_node4) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t2] Registered new searcher Searcher@5342bca4[tlog_replica_test_delete_by_id_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(7.6.0):C1))}
[junit4] 2> 450687 INFO (qtp1675747423-3008) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4
[junit4] 2> 450688 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={_stateVer_=tlog_replica_test_delete_by_id:6&commit=true&wt=javabin&version=2}{commit=} 0 6
[junit4] 2> 450692 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 450793 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 450894 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 450995 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451096 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451196 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451266 INFO (OverseerCollectionConfigSetProcessor-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 451297 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451398 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451498 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451599 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451700 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451800 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 451901 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452002 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452103 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452203 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452304 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452405 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452506 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452590 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/
[junit4] 2> 452592 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 452592 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 452592 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 1543193637439
[junit4] 2> 452592 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 452592 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 452593 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 452593 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=0
[junit4] 2> 452593 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Number of files in latest index in master: 15
[junit4] 2> 452594 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@38e209ad lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@767d8fad)
[junit4] 2> 452595 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452596 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452596 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452597 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452605 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452605 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452606 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452606 INFO (qtp706330176-3004) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 452606 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452607 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.tix&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452608 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452608 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452609 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.tbk&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452609 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452609 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452610 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 452610 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Bytes downloaded: 2354, Bytes skipped downloading: 0
[junit4] 2> 452610 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=2354) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@38e209ad lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@767d8fad)
[junit4] 2> 452612 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 452613 INFO (indexFetcher-1519-thread-1) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3981d26d[tlog_replica_test_delete_by_id_shard1_replica_t1] main]
[junit4] 2> 452614 INFO (searcherExecutor-1508-thread-1) [ ] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t1] Registered new searcher Searcher@3981d26d[tlog_replica_test_delete_by_id_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(7.6.0):C1))}
[junit4] 2> 452707 INFO (qtp706330176-3017) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 452711 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 452714 INFO (qtp706330176-3016) [n:127.0.0.1:39937_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node3 x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:39227/solr/tlog_replica_test_delete_by_id_shard1_replica_t2/&wt=javabin&version=2}{delete=[1 (-1618155813699846144)]} 0 0
[junit4] 2> 452714 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1 (-1618155813699846144)]} 0 1
[junit4] 2> 452718 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr c:tlog_replica_test_delete_by_id s:shard1 r:core_node4 x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_delete_by_id_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 452719 INFO (TEST-TestTlogReplica.testDeleteById-seed#[C1E37A5B14169792]) [ ] o.a.s.c.TestTlogReplica tearDown deleting collection
[junit4] 2> 452719 INFO (qtp1675747423-3076) [n:127.0.0.1:39227_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=tlog_replica_test_delete_by_id&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 452721 INFO (OverseerThreadFactory-1496-thread-2-processing-n:127.0.0.1:39227_solr) [n:127.0.0.1:39227_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true&deleteMetricsHistory=true, asyncId=null
[junit4] 2> 452723 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2, tag=null
[junit4] 2> 452723 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1, tag=null
[junit4] 2> 452723 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3c24d6df: rootName = solr_39227, domain = solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2, service url = null, agent id = null] for registry solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2 / com.codahale.metrics.MetricRegistry@179cda2e
[junit4] 2> 452723 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f3a5167: rootName = solr_39937, domain = solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1, service url = null, agent id = null] for registry solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1 / com.codahale.metrics.MetricRegistry@2d36b0b2
[junit4] 2> 452731 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_delete_by_id_shard1_replica_t1 stopping background replication from leader
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_delete_by_id_shard1_replica_t2 stopping background replication from leader
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t2] CLOSING SolrCore org.apache.solr.core.SolrCore@7d0715fe
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_delete_by_id.shard1.replica_t2, tag=7d0715fe
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tlog_replica_test_delete_by_id.shard1.leader, tag=7d0715fe
[junit4] 2> 452732 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_delete_by_id_shard1_replica_t1] CLOSING SolrCore org.apache.solr.core.SolrCore@32b4df27
[junit4] 2> 452732 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_delete_by_id.shard1.replica_t1, tag=32b4df27
[junit4] 2> 452732 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tlog_replica_test_delete_by_id.shard1.leader, tag=32b4df27
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
[junit4] 2> 452732 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
[junit4] 2> 452732 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@221bf366 commitCommandVersion:0
[junit4] 2> 452732 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@27cedbc2 commitCommandVersion:0
[junit4] 2> 452736 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_delete_by_id/terms/shard1 to Terms{values={core_node4=1}, version=3}
[junit4] 2> 452736 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
[junit4] 2> 452736 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
[junit4] 2> 452736 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_delete_by_id/terms/shard1 to Terms{values={}, version=4}
[junit4] 2> 452742 INFO (qtp706330176-3018) [n:127.0.0.1:39937_solr x:tlog_replica_test_delete_by_id_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&deleteMetricsHistory=true&core=tlog_replica_test_delete_by_id_shard1_replica_t1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 452742 WARN (zkCallback-784-thread-1) [ ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
[junit4] 2> 452743 INFO (zkCallback-770-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_delete_by_id/state.json] for collection [tlog_replica_test_delete_by_id] has occurred - updating... (live nodes size: [2])
[junit4] 2> 452743 INFO (zkCallback-784-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_delete_by_id/state.json] for collection [tlog_replica_test_delete_by_id] has occurred - updating... (live nodes size: [2])
[junit4] 2> 452743 INFO (qtp1675747423-3006) [n:127.0.0.1:39227_solr x:tlog_replica_test_delete_by_id_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&deleteMetricsHistory=true&core=tlog_replica_test_delete_by_id_shard1_replica_t2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=20
[junit4] 2> 453457 INFO (qtp1675747423-3076) [n:127.0.0.1:39227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=tlog_replica_test_delete_by_id&action=DELETE&wt=javabin&version=2} status=0 QTime=737
[junit4] 2> 453457 INFO (TEST-TestTlogReplica.testDeleteById-seed#[C1E37A5B14169792]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDeleteById
[junit4] 2> 453465 INFO (TEST-TestTlogReplica.testKillLeader-seed#[C1E37A5B14169792]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testKillLeader
[junit4] 2> 453465 INFO (qtp1675747423-3080) [n:127.0.0.1:39227_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_kill_leader&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 453467 INFO (OverseerThreadFactory-1496-thread-3-processing-n:127.0.0.1:39227_solr) [n:127.0.0.1:39227_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_kill_leader
[junit4] 2> 453467 INFO (OverseerCollectionConfigSetProcessor-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 453672 INFO (qtp1675747423-3007) [n:127.0.0.1:39227_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 453674 INFO (qtp706330176-3081) [n:127.0.0.1:39937_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 453678 INFO (OverseerStateUpdate-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_kill_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_kill_leader_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39937/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 453678 INFO (OverseerStateUpdate-72087609322831876-127.0.0.1:39227_solr-n_0000000000) [n:127.0.0.1:39227_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"tlog_replica_test_kill_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"tlog_replica_test_kill_leader_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39227/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 453
[...truncated too long message...]
y-closer-134-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2008c642: rootName = solr_45863, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@13d983f3
[junit4] 2> 212918 INFO (jetty-closer-134-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45863_solr
[junit4] 2> 212919 INFO (jetty-closer-134-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 212919 INFO (zkCallback-183-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 212919 INFO (zkCallback-190-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 212919 INFO (zkCallback-261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 212919 INFO (zkCallback-254-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 212920 INFO (jetty-closer-134-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72087823404040205-127.0.0.1:45863_solr-n_0000000002) closing
[junit4] 2> 212920 INFO (OverseerStateUpdate-72087823404040205-127.0.0.1:45863_solr-n_0000000002) [n:127.0.0.1:45863_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45863_solr
[junit4] 2> 212920 WARN (OverseerAutoScalingTriggerThread-72087823404040205-127.0.0.1:45863_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 212922 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1001b7e52a1001d, likely client has closed socket
[junit4] 2> 212923 INFO (jetty-closer-134-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@77cd8e8e{/solr,null,UNAVAILABLE}
[junit4] 2> 212923 INFO (jetty-closer-134-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 212923 INFO (jetty-closer-134-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@dc3919{/solr,null,UNAVAILABLE}
[junit4] 2> 212923 INFO (jetty-closer-134-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 212924 ERROR (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 212924 INFO (SUITE-TestTlogReplica-seed#[C1E37A5B14169792]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45469 45469
[junit4] 2> 212965 INFO (Thread-98) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45469 45469
[junit4] 2> 212966 WARN (Thread-98) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 32 /solr/configs/conf
[junit4] 2> 31 /solr/collections/tlog_replica_test_create_delete/terms/shard1
[junit4] 2> 30 /solr/collections/tlog_replica_test_create_delete/terms/shard2
[junit4] 2> 19 /solr/collections/tlog_replica_test_recovery/terms/shard1
[junit4] 2> 19 /solr/collections/tlog_replica_test_remove_leader/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_kill_leader/terms/shard1
[junit4] 2> 16 /solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
[junit4] 2> 15 /solr/aliases.json
[junit4] 2> 14 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
[junit4] 2> 14 /solr/collections/tlog_replica_test_add_docs/terms/shard1
[junit4] 2> 13 /solr/collections/tlog_replica_test_real_time_get/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_delete_by_id/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
[junit4] 2> 8 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
[junit4] 2> 7 /solr/security.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 30 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
[junit4] 2> 22 /solr/collections/tlog_replica_test_create_delete/state.json
[junit4] 2> 16 /solr/collections/tlog_replica_test_kill_leader/state.json
[junit4] 2> 15 /solr/clusterstate.json
[junit4] 2> 15 /solr/clusterprops.json
[junit4] 2> 11 /solr/collections/tlog_replica_test_recovery/state.json
[junit4] 2> 9 /solr/collections/tlog_replica_test_remove_leader/state.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
[junit4] 2> 7 /solr/collections/tlog_replica_test_basic_leader_election/state.json
[junit4] 2> 5 /solr/collections/tlog_replica_test_add_docs/state.json
[junit4] 2> 5 /solr/collections/tlog_replica_test_real_time_get/state.json
[junit4] 2> 4 /solr/overseer_elect/election/72087823404040205-127.0.0.1:45863_solr-n_0000000002
[junit4] 2> 3 /solr/autoscaling.json
[junit4] 2> 3 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
[junit4] 2> 3 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
[junit4] 2> 3 /solr/collections/tlog_replica_test_delete_by_id/state.json
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72087823404040217-core_node3-n_0000000001
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72087823404040217-core_node16-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72087823404040217-core_node9-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72087823404040205-core_node4-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/live_nodes
[junit4] 2> 15 /solr/collections
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 3 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 3 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2> 3 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_C1E37A5B14169792-002
[junit4] 2> Nov 26, 2018 1:50:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 6 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {foo=PostingsFormat(name=LuceneVarGapDocFreqInterval), title_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=FSTOrd50, foo_s=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene70), inplace_updatable_int=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1817, maxMBSortInHeap=5.344373049261223, sim=RandomSimilarity(queryNorm=false): {}, locale=kn-IN, timezone=Asia/Vladivostok
[junit4] 2> NOTE: Linux 4.15.0-39-generic amd64/Oracle Corporation 9.0.4 (64-bit)/cpus=8,threads=1,free=370092848,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestTlogReplica, TestTlogReplica]
[junit4] Completed [5/5 (5!)] on J1 in 104.61s, 13 tests, 1 failure <<< FAILURES!
[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 65 tests, 5 failures [seed: C1E37A5B14169792]
Total time: 3 minutes 36 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=ad90d8bb44d9470d2f4162642d1aeffb176d6956, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.6-Linux
[WARNINGS] Computing warning deltas based on reference build #28
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[JENKINS] Lucene-Solr-7.6-Linux (32bit/jdk1.8.0_172) - Build # 28 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.6-Linux/28/
Java: 32bit/jdk1.8.0_172 -server -XX:+UseSerialGC
6 tests failed.
FAILED: org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:32801_solr, 127.0.0.1:33451_solr, 127.0.0.1:40799_solr] Last available state: DocCollection(raceDeleteReplica_true//collections/raceDeleteReplica_true/state.json/13)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"raceDeleteReplica_true_shard1_replica_n2", "base_url":"https://127.0.0.1:45853/solr", "node_name":"127.0.0.1:45853_solr", "state":"down", "type":"NRT", "leader":"true"}, "core_node6":{ "core":"raceDeleteReplica_true_shard1_replica_n5", "base_url":"https://127.0.0.1:45853/solr", "node_name":"127.0.0.1:45853_solr", "state":"down", "type":"NRT"}, "core_node3":{ "core":"raceDeleteReplica_true_shard1_replica_n1", "base_url":"https://127.0.0.1:40799/solr", "node_name":"127.0.0.1:40799_solr", "state":"down", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:32801_solr, 127.0.0.1:33451_solr, 127.0.0.1:40799_solr]
Last available state: DocCollection(raceDeleteReplica_true//collections/raceDeleteReplica_true/state.json/13)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"raceDeleteReplica_true_shard1_replica_n2",
"base_url":"https://127.0.0.1:45853/solr",
"node_name":"127.0.0.1:45853_solr",
"state":"down",
"type":"NRT",
"leader":"true"},
"core_node6":{
"core":"raceDeleteReplica_true_shard1_replica_n5",
"base_url":"https://127.0.0.1:45853/solr",
"node_name":"127.0.0.1:45853_solr",
"state":"down",
"type":"NRT"},
"core_node3":{
"core":"raceDeleteReplica_true_shard1_replica_n1",
"base_url":"https://127.0.0.1:40799/solr",
"node_name":"127.0.0.1:40799_solr",
"state":"down",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:F4B0D27A5925DBDE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:334)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:34181_solr, 127.0.0.1:34619_solr, 127.0.0.1:41089_solr] Last available state: DocCollection(raceDeleteReplica_true//collections/raceDeleteReplica_true/state.json/14)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node3":{ "core":"raceDeleteReplica_true_shard1_replica_n1", "base_url":"https://127.0.0.1:41729/solr", "node_name":"127.0.0.1:41729_solr", "state":"down", "type":"NRT", "leader":"true"}, "core_node6":{ "core":"raceDeleteReplica_true_shard1_replica_n5", "base_url":"https://127.0.0.1:41729/solr", "node_name":"127.0.0.1:41729_solr", "state":"down", "type":"NRT"}, "core_node4":{ "core":"raceDeleteReplica_true_shard1_replica_n2", "base_url":"https://127.0.0.1:34181/solr", "node_name":"127.0.0.1:34181_solr", "state":"down", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:34181_solr, 127.0.0.1:34619_solr, 127.0.0.1:41089_solr]
Last available state: DocCollection(raceDeleteReplica_true//collections/raceDeleteReplica_true/state.json/14)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node3":{
"core":"raceDeleteReplica_true_shard1_replica_n1",
"base_url":"https://127.0.0.1:41729/solr",
"node_name":"127.0.0.1:41729_solr",
"state":"down",
"type":"NRT",
"leader":"true"},
"core_node6":{
"core":"raceDeleteReplica_true_shard1_replica_n5",
"base_url":"https://127.0.0.1:41729/solr",
"node_name":"127.0.0.1:41729_solr",
"state":"down",
"type":"NRT"},
"core_node4":{
"core":"raceDeleteReplica_true_shard1_replica_n2",
"base_url":"https://127.0.0.1:34181/solr",
"node_name":"127.0.0.1:34181_solr",
"state":"down",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:F4B0D27A5925DBDE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:334)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10006_solr, 127.0.0.1:10008_solr, 127.0.0.1:10005_solr, 127.0.0.1:10009_solr, 127.0.0.1:10007_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10008_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10006_solr, 127.0.0.1:10008_solr, 127.0.0.1:10005_solr, 127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10008_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:1E86D684209479B2]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 27
live nodes:[127.0.0.1:10006_solr, 127.0.0.1:10008_solr, 127.0.0.1:10005_solr, 127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10008_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10008_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 41 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 127.0.0.1:10014_solr, 127.0.0.1:10013_solr, 127.0.0.1:10011_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10014_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 127.0.0.1:10014_solr, 127.0.0.1:10013_solr, 127.0.0.1:10011_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10014_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:1E86D684209479B2]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 25
live nodes:[127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 127.0.0.1:10014_solr, 127.0.0.1:10013_solr, 127.0.0.1:10011_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10014_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10014_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 41 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10017_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10019_solr, 127.0.0.1:10015_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10019_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10017_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10019_solr, 127.0.0.1:10015_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10019_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:1E86D684209479B2]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 25
live nodes:[127.0.0.1:10017_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10019_solr, 127.0.0.1:10015_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10019_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10019_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 41 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica
Error Message:
Timeout waiting for collection to become active Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr] Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={ "replicationFactor":"1", "pullReplicas":"0", "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0", "autoCreated":"true", "policy":"c1", "shards":{"shard1":{ "replicas":{"core_node1":{ "core":"testCreateCollectionAddReplica_shard1_replica_n1", "SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":10240, "node_name":"127.0.0.1:10002_solr", "state":"active", "type":"NRT", "INDEX.sizeInGB":9.5367431640625E-6, "SEARCHER.searcher.numDocs":0}}, "range":"80000000-7fffffff", "state":"active"}}}
Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
Last available state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at __randomizedtesting.SeedInfo.seed([9EA6B3AA31D79114:1E86D684209479B2]:0)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last ClusterState: znodeVersion: 25
live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
collections:{testCreateCollectionAddReplica=DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}}, last coll state: DocCollection(testCreateCollectionAddReplica//clusterstate.json/24)={
"replicationFactor":"1",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0",
"autoCreated":"true",
"policy":"c1",
"shards":{"shard1":{
"replicas":{"core_node1":{
"core":"testCreateCollectionAddReplica_shard1_replica_n1",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":10240,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"INDEX.sizeInGB":9.5367431640625E-6,
"SEARCHER.searcher.numDocs":0}},
"range":"80000000-7fffffff",
"state":"active"}}}
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:112)
at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:64)
... 41 more
Build Log:
[...truncated 12647 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[junit4] 2> 101735 INFO (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_9EA6B3AA31D79114-001/init-core-data-001
[junit4] 2> 101736 WARN (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 101736 INFO (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 101738 INFO (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 101759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 101759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 101759 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 101760 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 101760 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 101764 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 101764 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 101769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 101769 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 101769 DEBUG (ScheduledTrigger-209-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 101770 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 101770 INFO (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 101774 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDataProviderPerReplicaDetails
[junit4] 2> 101774 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 101774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 101775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 101775 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 101775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 101775 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 101775 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 101776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 101776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 101776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 101776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 101777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 101777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 101777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 101777 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101798 DEBUG (ScheduledTrigger-214-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101818 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101841 DEBUG (ScheduledTrigger-214-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101861 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101881 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101902 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101922 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101942 DEBUG (ScheduledTrigger-214-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101962 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101977 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection perReplicaDataColl, currentVersion=1
[junit4] 2> 101978 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
[junit4] 2> 101978 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
[junit4] 2> 101983 DEBUG (ScheduledTrigger-214-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 101989 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
[junit4] 2> 101991 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
[junit4] 2> 101995 DEBUG (simCloudManagerPool-213-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection perReplicaDataColl, currentVersion=3
[junit4] 2> 102003 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102007 DEBUG (simCloudManagerPool-213-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
[junit4] 2> 102007 DEBUG (simCloudManagerPool-213-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
[junit4] 2> 102010 DEBUG (simCloudManagerPool-213-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for perReplicaDataColl / shard1 (currentVersion=4): {"core_node1":{
[junit4] 2> "core":"perReplicaDataColl_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"perReplicaDataColl",
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 102019 DEBUG (simCloudManagerPool-213-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
[junit4] 2> 102019 DEBUG (simCloudManagerPool-213-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
[junit4] 2> 102023 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102029 DEBUG (simCloudManagerPool-213-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
[junit4] 2> 102030 DEBUG (simCloudManagerPool-213-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
[junit4] 2> 102040 DEBUG (simCloudManagerPool-213-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
[junit4] 2> 102040 DEBUG (simCloudManagerPool-213-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
[junit4] 2> 102043 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102050 DEBUG (simCloudManagerPool-213-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
[junit4] 2> 102051 DEBUG (simCloudManagerPool-213-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
[junit4] 2> 102051 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
[junit4] 2> 102051 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
[junit4] 2> 102064 DEBUG (ScheduledTrigger-214-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102084 DEBUG (ScheduledTrigger-214-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102104 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102124 DEBUG (ScheduledTrigger-214-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102145 DEBUG (ScheduledTrigger-214-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102165 DEBUG (ScheduledTrigger-214-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102165 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDataProviderPerReplicaDetails
[junit4] 2> 102166 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
[junit4] 2> 102166 DEBUG (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
[junit4] 2> 102167 INFO (TEST-TestSimPolicyCloud.testDataProviderPerReplicaDetails-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 5
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETE 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 102169 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSplitShard
[junit4] 2> 102169 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 102169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 102170 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 102170 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 102170 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 102170 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 102170 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 102171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 102171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 102171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
[junit4] 2> 102171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 102172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 102173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 102173 DEBUG (ScheduledTrigger-217-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102194 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102215 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102235 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102255 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102275 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102295 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102316 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102336 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102356 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102376 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102382 DEBUG (simCloudManagerPool-216-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
[junit4] 2> 102382 DEBUG (simCloudManagerPool-216-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
[junit4] 2> 102382 DEBUG (simCloudManagerPool-216-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 102384 DEBUG (simCloudManagerPool-216-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 102384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 102384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 102384 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionSplitShard, currentVersion=11
[junit4] 2> 102385 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
[junit4] 2> 102385 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
[junit4] 2> 102395 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
[junit4] 2> 102395 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
[junit4] 2> 102396 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102401 DEBUG (simCloudManagerPool-216-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionSplitShard, currentVersion=13
[junit4] 2> 102411 DEBUG (simCloudManagerPool-216-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
[junit4] 2> 102411 DEBUG (simCloudManagerPool-216-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
[junit4] 2> 102412 DEBUG (simCloudManagerPool-216-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1 (currentVersion=14): {"core_node1":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 102417 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102422 DEBUG (simCloudManagerPool-216-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
[junit4] 2> 102422 DEBUG (simCloudManagerPool-216-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
[junit4] 2> 102422 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
[junit4] 2> 102422 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
[junit4] 2> 102437 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102457 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102477 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102498 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102518 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102523 INFO (simCloudManagerPool-216-thread-6) [ ] o.a.s.c.a.c.SplitShardCmd numSubShards set at: 2
[junit4] 2> 102538 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102558 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102569 DEBUG (simCloudManagerPool-216-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
[junit4] 2> 102570 DEBUG (simCloudManagerPool-216-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
[junit4] 2> 102571 DEBUG (simCloudManagerPool-216-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_0 (currentVersion=17): {"core_node6":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_0_replica_n5",
[junit4] 2> "shard":"shard1_0",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":20480,
[junit4] 2> "base_url":"http://127.0.0.1:10000/solr",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":1.9073486328125E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 102571 DEBUG (simCloudManagerPool-216-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionSplitShard / shard1_1 (currentVersion=17): {"core_node10":{
[junit4] 2> "core":"testCreateCollectionSplitShard_shard1_1_replica_n9",
[junit4] 2> "shard":"shard1_1",
[junit4] 2> "collection":"testCreateCollectionSplitShard",
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "base_url":"http://127.0.0.1:10004/solr",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 102572 DEBUG (simCloudManagerPool-216-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
[junit4] 2> 102572 DEBUG (simCloudManagerPool-216-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
[junit4] 2> 102573 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
[junit4] 2> 102573 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
[junit4] 2> 102578 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102599 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102619 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102639 DEBUG (ScheduledTrigger-217-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102659 DEBUG (ScheduledTrigger-217-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102674 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSplitShard
[junit4] 2> 102674 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
[junit4] 2> 102674 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
[junit4] 2> 102674 INFO (TEST-TestSimPolicyCloud.testCreateCollectionSplitShard-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 5
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - DELETE 1
[junit4] 2> ## - SPLITSHARD 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] 2> 102677 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionAddReplica
[junit4] 2> 102677 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 102677 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 102677 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 102678 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 102678 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 102678 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 102678 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 102678 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 102679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 102679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 102679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 102680 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 102680 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102680 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 102680 DEBUG (ScheduledTrigger-220-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102701 DEBUG (ScheduledTrigger-220-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102721 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102741 DEBUG (ScheduledTrigger-220-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102761 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102781 DEBUG (ScheduledTrigger-220-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102801 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102821 DEBUG (ScheduledTrigger-220-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102842 DEBUG (ScheduledTrigger-220-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102862 DEBUG (ScheduledTrigger-220-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102879 DEBUG (simCloudManagerPool-219-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
[junit4] 2> 102879 DEBUG (simCloudManagerPool-219-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
[junit4] 2> 102879 DEBUG (simCloudManagerPool-219-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 102879 DEBUG (simCloudManagerPool-219-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 102880 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 102880 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 102880 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 102880 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCreateCollectionAddReplica, currentVersion=21
[junit4] 2> 102881 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
[junit4] 2> 102881 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
[junit4] 2> 102882 DEBUG (ScheduledTrigger-220-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102891 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
[junit4] 2> 102891 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
[junit4] 2> 102892 DEBUG (simCloudManagerPool-219-thread-2) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCreateCollectionAddReplica, currentVersion=23
[junit4] 2> 102902 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102903 DEBUG (simCloudManagerPool-219-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
[junit4] 2> 102903 DEBUG (simCloudManagerPool-219-thread-3) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
[junit4] 2> 102904 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
[junit4] 2> 102904 DEBUG (simCloudManagerPool-219-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCreateCollectionAddReplica / shard1 (currentVersion=24): {"core_node1":{
[junit4] 2> "core":"testCreateCollectionAddReplica_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testCreateCollectionAddReplica",
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 102904 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddReplica-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
[junit4] 2> 102923 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102943 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102963 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 102983 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 103004 DEBUG (ScheduledTrigger-220-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 103024 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 103044 DEBUG (ScheduledTrigger-220-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 103064 DEBUG (ScheduledTrigger-220-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLost
[...truncated too long message...]
o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 55
[junit4] 2> 165560 DEBUG (ScheduledTrigger-300-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165581 DEBUG (ScheduledTrigger-300-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165601 DEBUG (ScheduledTrigger-300-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165621 DEBUG (ScheduledTrigger-300-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165641 DEBUG (ScheduledTrigger-300-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165653 INFO (simCloudManagerPool-299-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10016_solr for creating new replica of shard s3 for collection policiesTest
[junit4] 2> 165661 DEBUG (ScheduledTrigger-300-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 165663 DEBUG (simCloudManagerPool-299-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=56
[junit4] 2> 165664 DEBUG (simCloudManagerPool-299-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 56
[junit4] 2> 165664 DEBUG (simCloudManagerPool-299-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=57
[junit4] 2> 165664 DEBUG (simCloudManagerPool-299-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 57
[junit4] 2> 165664 DEBUG (simCloudManagerPool-299-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for policiesTest / s3 (currentVersion=57): {"core_node15":{
[junit4] 2> "core":"policiesTest_s3_replica_n14",
[junit4] 2> "shard":"s3",
[junit4] 2> "collection":"policiesTest",
[junit4] 2> "node_name":"127.0.0.1:10016_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6}}
[junit4] 2> 165665 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=58
[junit4] 2> 165665 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 58
[junit4] 2> 165665 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionAddShardUsingPolicy
[junit4] 2> 165665 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=59
[junit4] 2> 165666 DEBUG (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 59
[junit4] 2> 165666 INFO (TEST-TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy-seed#[9EA6B3AA31D79114]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 5
[junit4] 2> ## Empty nodes: 4
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * policiesTest
[junit4] 2> ## shardsTotal 3
[junit4] 2> ## shardsState {active=3}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 3
[junit4] 2> ## activeReplicas 3
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 30,720
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 3
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 3
[junit4] 2> ## * policiesTest 3
[junit4] 2> ## - active 3
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - CREATESHARD 1
[junit4] 2> ## - autoscaling 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2>
[junit4] OK 0.35s J2 | TestSimPolicyCloud.testCreateCollectionAddShardUsingPolicy
[junit4] 2> 165667 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 165667 DEBUG (SUITE-TestSimPolicyCloud-seed#[9EA6B3AA31D79114]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.6-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimPolicyCloud_9EA6B3AA31D79114-003
[junit4] 2> Nov 25, 2018 8:46:16 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 5 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1295, maxMBSortInHeap=5.156309467628896, sim=RandomSimilarity(queryNorm=true): {}, locale=es-ES, timezone=Asia/Tel_Aviv
[junit4] 2> NOTE: Linux 4.15.0-39-generic i386/Oracle Corporation 1.8.0_172 (32-bit)/cpus=8,threads=1,free=100465232,total=194736128
[junit4] 2> NOTE: All tests run in this JVM: [TestSimPolicyCloud, TestSimPolicyCloud, TestSimPolicyCloud, DeleteReplicaTest, TestSimPolicyCloud]
[junit4] Completed [7/10 (4!)] on J2 in 4.41s, 7 tests, 1 failure <<< FAILURES!
[...truncated 44 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.6-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 70 tests, 4 failures [seed: 9EA6B3AA31D79114]
Total time: 3 minutes 0 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 1/5 failed: org.apache.solr.cloud.DeleteReplicaTest
[repro] 3/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=ad90d8bb44d9470d2f4162642d1aeffb176d6956, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.6-Linux
[WARNINGS] Computing warning deltas based on reference build #27
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2