You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2019/02/21 13:45:07 UTC
[JENKINS-EA] Lucene-Solr-8.x-Linux
(64bit/jdk-12-ea+shipilev-fastdebug) - Build # 193 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/193/
Java: 64bit/jdk-12-ea+shipilev-fastdebug -XX:+UseCompressedOops -XX:+UseG1GC
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([14F063EA3FAE6DDC:DD45214436C9AB29]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testEventQueue(TestSimTriggerIntegration.java:757)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([14F063EA3FAE6DDC:DD45214436C9AB29]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testEventQueue(TestSimTriggerIntegration.java:757)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate
Error Message:
The trigger did not start in time
Stack Trace:
java.lang.AssertionError: The trigger did not start in time
at __randomizedtesting.SeedInfo.seed([14F063EA3FAE6DDC:49B87D63F068CB93]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration.testSearchRate(TestSimTriggerIntegration.java:1369)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)
Build Log:
[...truncated 14616 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
[junit4] 2> 3055380 INFO (SUITE-TestSimTriggerIntegration-seed#[14F063EA3FAE6DDC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_14F063EA3FAE6DDC-001/init-core-data-001
[junit4] 2> 3055381 INFO (SUITE-TestSimTriggerIntegration-seed#[14F063EA3FAE6DDC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 3055381 INFO (SUITE-TestSimTriggerIntegration-seed#[14F063EA3FAE6DDC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 3055384 INFO (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 3055387 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10244_solr
[junit4] 2> 3055387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3055387 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3055387 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3055387 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3055387 INFO (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3055388 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3055388 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3055388 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10245_solr, 127.0.0.1:10244_solr]
[junit4] 2> 3055388 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055388 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3055388 DEBUG (simCloudManagerPool-8300-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055388 DEBUG (ScheduledTrigger-8301-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055388 DEBUG (simCloudManagerPool-8300-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3055395 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3055395 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055395 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3055434 DEBUG (ScheduledTrigger-8301-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055435 DEBUG (simCloudManagerPool-8300-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055435 DEBUG (simCloudManagerPool-8300-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3055435 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3055435 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3055435 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10245_solr, 127.0.0.1:10244_solr]
[junit4] 2> 3055436 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055436 DEBUG (ScheduledTrigger-8301-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055436 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3055441 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10245_solr
[junit4] 2> 3055454 DEBUG (ScheduledTrigger-8301-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 2
[junit4] 2> 3055454 DEBUG (ScheduledTrigger-8301-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10245_solr
[junit4] 2> 3055458 DEBUG (ScheduledTrigger-8301-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1 and last live nodes: 2
[junit4] 2> 3055458 DEBUG (ScheduledTrigger-8301-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10245_solr
[junit4] 2> 3055474 DEBUG (ScheduledTrigger-8301-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055478 DEBUG (ScheduledTrigger-8301-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055495 DEBUG (ScheduledTrigger-8301-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055498 DEBUG (ScheduledTrigger-8301-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055515 DEBUG (ScheduledTrigger-8301-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055518 DEBUG (ScheduledTrigger-8301-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 3055527 DEBUG (ScheduledTrigger-8301-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10245_solr]
[junit4] 2> 3055528 DEBUG (ScheduledTrigger-8301-thread-3) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f4147d12d2T7rabm5m7xhngeaxopjwse74uo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":465042222682834,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465042222682834],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465045708196434,
[junit4] 2> "nodeNames":["127.0.0.1:10245_solr"]}}
[junit4] 2> 3055528 DEBUG (AutoscalingActionExecutor-8302-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f4147d12d2T7rabm5m7xhngeaxopjwse74uo",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":465042222682834,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465042222682834],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465045708196434,
[junit4] 2> "nodeNames":["127.0.0.1:10245_solr"]}}
[junit4] 2> 3055529 DEBUG (simCloudManagerPool-8300-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055529 DEBUG (simCloudManagerPool-8300-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3055529 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3055530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3055575 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3055576 DEBUG (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3055577 INFO (TEST-TestSimTriggerIntegration.testNodeLostTrigger-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
[junit4] 2> 3055580 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
[junit4] 2> 3055582 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10246_solr
[junit4] 2> 3055583 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3055583 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3055583 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3055583 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3055583 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3055583 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3055583 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3055583 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10246_solr, 127.0.0.1:10247_solr]
[junit4] 2> 3055584 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055584 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3055584 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055584 DEBUG (simCloudManagerPool-8306-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055584 DEBUG (simCloudManagerPool-8306-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3055584 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3055585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3055585 DEBUG (simCloudManagerPool-8306-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055585 DEBUG (simCloudManagerPool-8306-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3055585 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3055586 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3055586 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:10246_solr, 127.0.0.1:10247_solr]
[junit4] 2> 3055586 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 3055586 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055586 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055586 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3055586 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3055622 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055622 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055622 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3055623 DEBUG (simCloudManagerPool-8306-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055623 DEBUG (simCloudManagerPool-8306-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3055623 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3055624 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055624 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3055629 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10248_solr
[junit4] 2> 3055642 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2
[junit4] 2> 3055642 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055642 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3055642 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10248_solr at time 465051455341684
[junit4] 2> 3055663 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3
[junit4] 2> 3055663 DEBUG (ScheduledTrigger-8307-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055663 DEBUG (ScheduledTrigger-8307-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3055663 DEBUG (ScheduledTrigger-8307-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10248_solr] added at times [465051455341684], now=465052474607284
[junit4] 2> 3055663 DEBUG (ScheduledTrigger-8307-thread-4) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f63acc4474T7rabm5m7xhngeaxopjwse74uq",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":465051455341684,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465051455341684],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465052493736634,
[junit4] 2> "nodeNames":["127.0.0.1:10248_solr"]}}
[junit4] 2> 3055664 DEBUG (ScheduledTrigger-8307-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10248_solr
[junit4] 2> 3055664 DEBUG (AutoscalingActionExecutor-8308-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f63acc4474T7rabm5m7xhngeaxopjwse74uq",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":465051455341684,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465051455341684],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465052493736634,
[junit4] 2> "nodeNames":["127.0.0.1:10248_solr"]}}
[junit4] 2> 3055693 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10249_solr
[junit4] 2> 3055793 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055793 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 3
[junit4] 2> 3055793 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 3055793 DEBUG (ScheduledTrigger-8307-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10249_solr at time 465058968953234
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 4
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10249_solr] added at times [465058968953234], now=465059983894284
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f7faa4e192T7rabm5m7xhngeaxopjwse74ur",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":465058968953234,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465058968953234],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465059990620334,
[junit4] 2> "nodeNames":["127.0.0.1:10249_solr"]}}
[junit4] 2> 3055813 DEBUG (ScheduledTrigger-8307-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10249_solr
[junit4] 2> 3055814 DEBUG (AutoscalingActionExecutor-8308-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f7faa4e192T7rabm5m7xhngeaxopjwse74ur",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":465058968953234,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465058968953234],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465059990620334,
[junit4] 2> "nodeNames":["127.0.0.1:10249_solr"]}}
[junit4] 2> 3055814 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3055815 DEBUG (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3055816 INFO (TEST-TestSimTriggerIntegration.testCooldown-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCooldown
[junit4] 2> 3055821 INFO (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 3055824 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10250_solr
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3055825 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3055825 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3055825 INFO (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10251_solr, 127.0.0.1:10250_solr]
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3055825 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055827 DEBUG (simCloudManagerPool-8312-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055827 DEBUG (simCloudManagerPool-8312-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3055830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3055869 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3055869 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055869 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3055870 DEBUG (simCloudManagerPool-8312-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055870 DEBUG (simCloudManagerPool-8312-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3055870 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3055871 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3055871 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10251_solr, 127.0.0.1:10250_solr]
[junit4] 2> 3055871 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestDummyAction}], enabled=true}
[junit4] 2> 3055871 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055871 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3055871 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3055871 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3055871 DEBUG (simCloudManagerPool-8312-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055871 DEBUG (simCloudManagerPool-8312-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3055872 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3055872 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055872 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3055872 DEBUG (simCloudManagerPool-8312-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3055873 DEBUG (simCloudManagerPool-8312-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 3055873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 3055873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3055873 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 3055878 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10252_solr
[junit4] 2> 3055889 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2
[junit4] 2> 3055891 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3055891 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3055891 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10252_solr at time 465063903914284
[junit4] 2> 3055910 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3
[junit4] 2> 3055912 DEBUG (ScheduledTrigger-8313-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3055912 DEBUG (ScheduledTrigger-8313-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3055912 DEBUG (ScheduledTrigger-8313-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10252_solr] added at times [465063903914284], now=465064921649434
[junit4] 2> 3055912 DEBUG (ScheduledTrigger-8313-thread-3) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f920ca692cT7rabm5m7xhngeaxopjwse74ut",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465063903914284,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465063903914284],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465064936993934,
[junit4] 2> "nodeNames":["127.0.0.1:10252_solr"]}}
[junit4] 2> 3055912 DEBUG (ScheduledTrigger-8313-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10252_solr
[junit4] 2> 3055985 DEBUG (AutoscalingActionExecutor-8314-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6f920ca692cT7rabm5m7xhngeaxopjwse74ut",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465063903914284,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465063903914284],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465064936993934,
[junit4] 2> "nodeNames":["127.0.0.1:10252_solr"]}}
[junit4] 2> 3055986 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10253_solr
[junit4] 2> 3056085 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 3
[junit4] 2> 3056085 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3056086 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 3056086 DEBUG (ScheduledTrigger-8313-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10253_solr at time 465073613224734
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 4
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10253_solr] added at times [465073613224734], now=465074628534934
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fb6382bb1eT7rabm5m7xhngeaxopjwse74uu",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465073613224734,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465073613224734],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465074636428534,
[junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}}
[junit4] 2> 3056106 DEBUG (ScheduledTrigger-8313-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10253_solr
[junit4] 2> 3056107 DEBUG (AutoscalingActionExecutor-8314-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fb6382bb1eT7rabm5m7xhngeaxopjwse74uu",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465073613224734,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465073613224734],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465074636428534,
[junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}}
[junit4] 2> 3056107 WARN (AutoscalingActionExecutor-8314-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.solr.cloud.autoscaling.TriggerActionException: Error processing action for trigger event: {
[junit4] 2> "id":"1a6fb6382bb1eT7rabm5m7xhngeaxopjwse74uu",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465073613224734,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465073613224734],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465074636428534,
[junit4] 2> "nodeNames":["127.0.0.1:10253_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:327) [java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestDummyAction.process(TestSimTriggerIntegration.java:1046) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:324) ~[java/:?]
[junit4] 2> ... 6 more
[junit4] 2> 3056108 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3056109 DEBUG (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3056110 INFO (TEST-TestSimTriggerIntegration.testListeners-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 3056114 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 3056118 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10254_solr
[junit4] 2> 3056118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3056118 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3056118 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3056119 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3056119 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3056119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3056119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3056119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10254_solr, 127.0.0.1:10255_solr]
[junit4] 2> 3056119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056120 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3056120 DEBUG (ScheduledTrigger-8319-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056120 DEBUG (simCloudManagerPool-8318-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056120 DEBUG (simCloudManagerPool-8318-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3056121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3056121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056121 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3056121 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== KILL OVERSEER 1
[junit4] 2> 3056122 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3056122 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3056123 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3056123 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimCloudManager = killing node 127.0.0.1:10254_solr
[junit4] 2> 3056123 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10254_solr
[junit4] 2> 3056123 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10255_solr
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 0
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10255_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10254_solr
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056124 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3056124 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056125 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10254_solr
[junit4] 2> 3056145 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056165 DEBUG (ScheduledTrigger-8324-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056185 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056206 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056226 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056246 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056260 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== ADD TRIGGERS
[junit4] 2> 3056267 DEBUG (ScheduledTrigger-8324-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056287 DEBUG (ScheduledTrigger-8324-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056296 INFO (simCloudManagerPool-8323-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056296 DEBUG (simCloudManagerPool-8323-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056296 DEBUG (simCloudManagerPool-8323-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3056297 INFO (simCloudManagerPool-8323-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056297 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3056297 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10255_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056297 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 3056297 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056297 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3056298 INFO (simCloudManagerPool-8323-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056298 DEBUG (ScheduledTrigger-8324-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3056298 DEBUG (ScheduledTrigger-8324-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3056298 DEBUG (simCloudManagerPool-8323-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056298 DEBUG (simCloudManagerPool-8323-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3056298 INFO (simCloudManagerPool-8323-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056298 INFO (simCloudManagerPool-8323-thread-2) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056298 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3056299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10255_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056299 DEBUG (ScheduledTrigger-8324-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3056303 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== ADD NODE 1
[junit4] 2> 3056304 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10257_solr
[junit4] 2> 3056304 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.TestSimTriggerIntegration ====== KILL OVERSEER 2
[junit4] 2> 3056304 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3056304 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3056305 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3056305 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimCloudManager = killing node 127.0.0.1:10255_solr
[junit4] 2> 3056305 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10255_solr
[junit4] 2> 3056305 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10257_solr
[junit4] 2> 3056310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3056310 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056310 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestEventMarkerAction init
[junit4] 2> 3056310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 0
[junit4] 2> 3056310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3056310 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10257_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10255_solr
[junit4] 2> 3056311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10257_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 3056311 DEBUG (ScheduledTrigger-8327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056311 DEBUG (ScheduledTrigger-8327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10255_solr
[junit4] 2> 3056311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10257_solr
[junit4] 2> 3056314 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10257_solr, 127.0.0.1:10256_solr]
[junit4] 2> 3056314 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10255_solr
[junit4] 2> 3056314 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3056314 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3056314 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10257_solr at time 465085021647034
[junit4] 2> 3056314 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056314 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10255_solr
[junit4] 2> 3056321 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056322 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3056332 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056332 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10255_solr]
[junit4] 2> 3056332 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fe0319f410T7rabm5m7xhngeaxopjwse74uw",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":465084880647184,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465084880647184],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465085931237434,
[junit4] 2> "nodeNames":["127.0.0.1:10255_solr"]}}
[junit4] 2> 3056334 DEBUG (AutoscalingActionExecutor-8328-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fe0319f410T7rabm5m7xhngeaxopjwse74uw",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":465084880647184,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465084880647184],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465085931237434,
[junit4] 2> "nodeNames":["127.0.0.1:10255_solr"]}}
[junit4] 2> 3056434 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3056434 DEBUG (ScheduledTrigger-8327-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056434 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3056434 DEBUG (ScheduledTrigger-8327-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056434 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10257_solr] added at times [465085021647034], now=465091057650134
[junit4] 2> 3056435 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fe0b8170baT7rabm5m7xhngeaxopjwse74ux",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465085021647034,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465085021647034],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465091065682084,
[junit4] 2> "nodeNames":["127.0.0.1:10257_solr"]}}
[junit4] 2> 3056435 DEBUG (ScheduledTrigger-8327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10257_solr
[junit4] 2> 3056435 DEBUG (AutoscalingActionExecutor-8328-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a6fe0b8170baT7rabm5m7xhngeaxopjwse74ux",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":465085021647034,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465085021647034],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465091065682084,
[junit4] 2> "nodeNames":["127.0.0.1:10257_solr"]}}
[junit4] 2> 3056465 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3056465 DEBUG (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3056466 INFO (TEST-TestSimTriggerIntegration.testNodeMarkersRegistration-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
[junit4] 2> 3056469 INFO (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 3056472 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10258_solr
[junit4] 2> 3056472 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3056472 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3056472 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3056472 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3056472 INFO (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3056472 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3056472 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3056474 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10259_solr, 127.0.0.1:10258_solr]
[junit4] 2> 3056478 DEBUG (simCloudManagerPool-8330-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056478 DEBUG (simCloudManagerPool-8330-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3056482 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056482 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056482 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 1
[junit4] 2> 3056482 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3056511 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
[junit4] 2> 3056511 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056511 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3056517 DEBUG (simCloudManagerPool-8330-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056517 DEBUG (simCloudManagerPool-8330-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3056517 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3056518 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3056518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10259_solr, 127.0.0.1:10258_solr]
[junit4] 2> 3056518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 3056518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3056518 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3056518 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3056518 DEBUG (simCloudManagerPool-8330-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056518 DEBUG (simCloudManagerPool-8330-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3056519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3056519 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3056519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10259_solr, 127.0.0.1:10258_solr]
[junit4] 2> 3056519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 3056519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3056519 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3056519 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3056524 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- creating marker: /autoscaling/nodeAdded/127.0.0.1:10260_solr
[junit4] 2> 3056531 DEBUG (ScheduledTrigger-8331-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 2
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10260_solr at time 465096273172784
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10260_solr] added at times [465096273172784], now=465096274608934
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a700aa261930T7rabm5m7xhngeaxopjwse74uz",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":465096273172784,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465096273172784],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465096293011834,
[junit4] 2> "nodeNames":["127.0.0.1:10260_solr"]}}
[junit4] 2> 3056539 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10260_solr
[junit4] 2> 3056540 INFO (AutoscalingActionExecutor-8332-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 3056540 DEBUG (AutoscalingActionExecutor-8332-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a700aa261930T7rabm5m7xhngeaxopjwse74uz",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":465096273172784,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465096273172784],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465096293011834,
[junit4] 2> "nodeNames":["127.0.0.1:10260_solr"]}}
[junit4] 2> 3056640 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3056645 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3056645 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3056640 DEBUG (ScheduledTrigger-8331-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3
[junit4] 2> 3056645 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3056646 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10260_solr at time 465101609975634
[junit4] 2> 3056646 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10260_solr] added at times [465101609975634], now=465101611920134
[junit4] 2> 3056646 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a701e83f3f52T7rabm5m7xhngeaxopjwse74v0",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":465101609975634,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465101609975634],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465101619421634,
[junit4] 2> "nodeNames":["127.0.0.1:10260_solr"]}}
[junit4] 2> 3056646 DEBUG (ScheduledTrigger-8331-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:10260_solr
[junit4] 2> 3056646 INFO (AutoscalingActionExecutor-8332-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration last action at 465096319233434 time = 465101638434634
[junit4] 2> 3056646 INFO (AutoscalingActionExecutor-8332-thread-1) [ ] o.a.s.c.a.s.TestSimTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 3056646 DEBUG (AutoscalingActionExecutor-8332-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"1a701e83f3f52T7rabm5m7xhngeaxopjwse74v0",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":465101609975634,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[465101609975634],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":465101619421634,
[junit4] 2> "nodeNames":["127.0.0.1:10260_solr"]}}
[junit4] 2> 3056647 DEBUG (simCloudManagerPool-8330-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056647 DEBUG (simCloudManagerPool-8330-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 3056647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 3056648 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3056648 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10259_solr, 127.0.0.1:10258_solr, 127.0.0.1:10260_solr]
[junit4] 2> 3056648 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056648 DEBUG (ScheduledTrigger-8331-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3 and last live nodes: 3
[junit4] 2> 3056648 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 3056648 DEBUG (simCloudManagerPool-8330-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3056649 DEBUG (simCloudManagerPool-8330-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 3056649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 3056649 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestSimTriggerIntegration TestTriggerAction init
[junit4] 2> 3056649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial livenodes: [127.0.0.1:10259_solr, 127.0.0.1:10258_solr, 127.0.0.1:10260_solr]
[junit4] 2> 3056649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 3056649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 3056649 DEBUG (ScheduledTrigger-8331-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3 and last live nodes: 3
[junit4] 2> 3056654 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider -- created marker: /autoscaling/nodeLost/127.0.0.1:10260_solr
[junit4] 2> 3056655 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2090) ~[?:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:201) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:835) [?:?]
[junit4] 2> 3056656 DEBUG (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3056658 INFO (TEST-TestSimTriggerIntegration.testTriggerThrottling-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 3056664 INFO (TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 3056667 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10261_solr
[junit4] 2> 3056667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3056667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3056667 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 3056667 DEBUG (TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 3056667 INFO (TEST-TestSimTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 3056667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
[junit4] 2> 3056667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3056668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10261_solr, 127.0.0.1:10262_solr]
[junit4] 2> 3056668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- cleaning old nodeLost / nodeAdded markers
[junit4] 2> 30
[...truncated too long message...]
-10) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
[junit4] 2> 20354 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10069_solr&type=NRT
[junit4] 2> 20361 INFO (simCloudManagerPool-152-thread-13) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10069_solr for creating new replica of shard shard1 for collection collection1
[junit4] 2> 20362 INFO (simCloudManagerPool-152-thread-13) [ ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
[junit4] 2> 20372 DEBUG (simCloudManagerPool-152-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
[junit4] 2> 20373 DEBUG (simCloudManagerPool-152-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
[junit4] 2> 20375 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10069_solr&type=NRT
[junit4] 2> 20376 INFO (simCloudManagerPool-152-thread-16) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10069_solr for creating new replica of shard shard1 for collection collection1
[junit4] 2> 20376 INFO (simCloudManagerPool-152-thread-16) [ ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
[junit4] 2> 20387 DEBUG (simCloudManagerPool-152-thread-16) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
[junit4] 2> 20387 DEBUG (simCloudManagerPool-152-thread-16) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
[junit4] 2> 20389 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
[junit4] 2> "id":"12dbc817a44acT7rabm5m7xhngeaxopjwse6y06",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":331762626086060,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "hotReplicas":[
[junit4] 2> {"core_node2":{
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "node_name":"127.0.0.1:10069_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "rate":250.0,
[junit4] 2> "state":"active",
[junit4] 2> "QUERY./select.requestTimes:1minRate":250,
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> {"core_node1":{
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "node_name":"127.0.0.1:10068_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "rate":250.0,
[junit4] 2> "state":"active",
[junit4] 2> "QUERY./select.requestTimes:1minRate":250,
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}],
[junit4] 2> "__start__":3,
[junit4] 2> "coldNodes":{},
[junit4] 2> "coldShards":{},
[junit4] 2> "coldReplicas":[],
[junit4] 2> "violationType":["hotShards"],
[junit4] 2> "coldCollections":{},
[junit4] 2> "hotShards":{"collection1":{"shard1":250.0}},
[junit4] 2> "hotCollections":{"collection1":500.0},
[junit4] 2> "hotNodes":{
[junit4] 2> "127.0.0.1:10069_solr":250.0,
[junit4] 2> "127.0.0.1:10068_solr":250.0},
[junit4] 2> "_enqueue_time_":331764853114710,
[junit4] 2> "requestedOps":[
[junit4] 2> {
[junit4] 2> "action":"ADDREPLICA",
[junit4] 2> "hints":{"COLL_SHARD":[{
[junit4] 2> "first":"collection1",
[junit4] 2> "second":"shard1"}]}},
[junit4] 2> {
[junit4] 2> "action":"ADDREPLICA",
[junit4] 2> "hints":{"COLL_SHARD":[{
[junit4] 2> "first":"collection1",
[junit4] 2> "second":"shard1"}]}},
[junit4] 2> {
[junit4] 2> "action":"ADDREPLICA",
[junit4] 2> "hints":{"COLL_SHARD":[{
[junit4] 2> "first":"collection1",
[junit4] 2> "second":"shard1"}]}}]}}
[junit4] 2> 20390 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: search_rate_trigger after 100ms
[junit4] 2> 20390 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 20390 DEBUG (AutoscalingActionExecutor-154-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 3865 ms for event id=12dbc817a44acT7rabm5m7xhngeaxopjwse6y06
[junit4] 2> 20392 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 20392 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 20393 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 20393 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 20393 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 20393 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 20393 DEBUG (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 20394 INFO (TEST-TestSimTriggerIntegration.testSearchRate-seed#[14F063EA3FAE6DDC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] OK 0.21s J0 | TestSimTriggerIntegration.testSearchRate
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimTriggerIntegration_14F063EA3FAE6DDC-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{}, maxPointsInLeafNode=880, maxMBSortInHeap=5.779864377444601, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6637b2cf), locale=ar-DJ, timezone=America/Aruba
[junit4] 2> NOTE: Linux 4.18.0-15-generic amd64/Aleksey Shipilev 12-testing (64-bit)/cpus=8,threads=1,free=30352040,total=91226112
[junit4] 2> NOTE: All tests run in this JVM: [TestSimTriggerIntegration, TestSimTriggerIntegration]
[junit4] Completed [5/5 (2!)] on J0 in 3.26s, 11 tests, 1 failure <<< FAILURES!
[...truncated 12 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-8.x-Linux/lucene/common-build.xml:1572: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-8.x-Linux/lucene/common-build.xml:1099: There were test failures: 5 suites, 55 tests, 2 failures [seed: 14F063EA3FAE6DDC]
Total time: 28 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 2/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimTriggerIntegration
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
No credentials specified
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=045dd2e26e170d77f9e727110962cfd22740f30b, workspace=/home/jenkins/workspace/Lucene-Solr-8.x-Linux
[WARNINGS] Computing warning deltas based on reference build #192
Recording test results
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2