You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/04/10 21:41:50 UTC
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1794 - Failure!
Error processing tokens: Error while parsing action 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input position (line 79, pos 4):
)"}
^
java.lang.OutOfMemoryError: Java heap space
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1795 - Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1795/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:654)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:654)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
Error Message:
action did not start
Stack Trace:
java.lang.AssertionError: action did not start
at __randomizedtesting.SeedInfo.seed([F64294E2F6294EBA:3FF7D64CFF4E884F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:640)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14144 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F64294E2F6294EBA-001/init-core-data-001
[junit4] 2> 3238810 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3238810 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testListeners
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 3238811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 3238811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeLostTriggerRestoreState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeLostTrigger
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testSearchRate
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testEventFromRestoredState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 3238812 DEBUG (ScheduledTrigger-11981-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testCooldown
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3238812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 3238823 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3238823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3238823 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3238830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3238830 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3238830 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_162]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_162]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
[junit4] 2> 3238830 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 3238831 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 3238831 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 3238831 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 3238831 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 3238831 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3238832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3238833 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3238833 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3238833 DEBUG (ScheduledTrigger-11984-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3238855 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3238875 DEBUG (ScheduledTrigger-11984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3238900 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3238920 DEBUG (ScheduledTrigger-11984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3238932 DEBUG (simCloudManagerPool-11983-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3238932 DEBUG (simCloudManagerPool-11983-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3238933 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3238935 DEBUG (simCloudManagerPool-11983-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3238935 DEBUG (simCloudManagerPool-11983-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3238935 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3238936 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 3238936 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3238936 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10022_solr at time 2479955644076439
[junit4] 2> 3238940 DEBUG (ScheduledTrigger-11984-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3238957 DEBUG (ScheduledTrigger-11984-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3238957 DEBUG (ScheduledTrigger-11984-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3238961 DEBUG (ScheduledTrigger-11984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3238977 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3238977 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3238981 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10022_solr] added at times [2479955644076439], now=2479958836909489
[junit4] 2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf81ab395197T71cpd1fni40admcqk3y2085dm",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":2479955644076439,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479955644076439],
[junit4] 2> "nodeNames":["127.0.0.1:10022_solr"]}}
[junit4] 2> 3239000 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 3239001 DEBUG (ScheduledTrigger-11984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10022_solr
[junit4] 2> 3239002 DEBUG (AutoscalingActionExecutor-11985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8cf81ab395197T71cpd1fni40admcqk3y2085dm",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":2479955644076439,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479955644076439],
[junit4] 2> "_enqueue_time_":2479958868013589,
[junit4] 2> "nodeNames":["127.0.0.1:10022_solr"]}}
[junit4] 2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 3239003 DEBUG (AutoscalingActionExecutor-11985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 98 ms for event id=8cf81ab395197T71cpd1fni40admcqk3y2085dm
[junit4] 2> 3239004 DEBUG (simCloudManagerPool-11983-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239004 DEBUG (simCloudManagerPool-11983-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 3239004 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 3239004 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 3239045 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 3239046 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
[junit4] 2> 3239047 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 3239047 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeMarkersRegistration
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 3239060 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 3239060 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3239060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 3239061 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3239060 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_162]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_162]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
[junit4] 2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 3239062 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239063 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 3239063 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239090 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239117 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239137 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239157 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239163 DEBUG (simCloudManagerPool-11986-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239163 DEBUG (simCloudManagerPool-11986-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 3239171 DEBUG (simCloudManagerPool-11986-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239171 DEBUG (simCloudManagerPool-11986-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 3239172 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 3239172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 3239172 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239172 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3239173 DEBUG (simCloudManagerPool-11986-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239173 DEBUG (simCloudManagerPool-11986-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 3239173 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3239173 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 3239174 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 3239174 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3239174 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3239180 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239200 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 2479968915485789
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10023_solr] added at times [2479968915485789], now=2479968917164989
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf84c242f45dT71cpd1fni40admcqk3y2085dp",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":2479968915485789,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479968915485789],
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239201 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 2479968929883589
[junit4] 2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10023_solr] added at times [2479968929883589], now=2479968931215039
[junit4] 2> 3239202 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085dq",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":2479968929883589,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479968929883589],
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239205 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 3239206 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
[junit4] 2> 3239206 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8cf84c242f45dT71cpd1fni40admcqk3y2085dp",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":2479968915485789,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479968915485789],
[junit4] 2> "_enqueue_time_":2479969084828039,
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239206 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 3239208 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 71 ms for event id=8cf84c242f45dT71cpd1fni40admcqk3y2085dp
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10023_solr] added at times [2479968929883589], now=2479974246723839
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085du",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":2479968929883589,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479968929883589],
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239308 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 3239309 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
[junit4] 2> 3239309 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8cf84c31ea5c5T71cpd1fni40admcqk3y2085du",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":2479968929883589,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479968929883589],
[junit4] 2> "_enqueue_time_":2479974261884939,
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239309 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 2479969176680839 time = 2479974314424839
[junit4] 2> 3239309 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 3239310 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 3239311 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 79 ms for event id=8cf84c31ea5c5T71cpd1fni40admcqk3y2085du
[junit4] 2> 3239311 DEBUG (simCloudManagerPool-11986-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239311 DEBUG (simCloudManagerPool-11986-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 3239311 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr]
[junit4] 2> 3239311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 3239311 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 3239312 DEBUG (simCloudManagerPool-11986-thread-10) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239312 DEBUG (simCloudManagerPool-11986-thread-10) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 3239313 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr]
[junit4] 2> 3239313 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
[junit4] 2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10023_solr]
[junit4] 2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf860ffc4707T71cpd1fni40admcqk3y2085dx",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":2479974514444039,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479974514444039],
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239313 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 3239314 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8cf860ffc4707T71cpd1fni40admcqk3y2085dx",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":2479974514444039,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479974514444039],
[junit4] 2> "_enqueue_time_":2479974521221589,
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239314 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 3239315 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 51 ms for event id=8cf860ffc4707T71cpd1fni40admcqk3y2085dx
[junit4] 2> 3239415 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239416 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 3239416 DEBUG (ScheduledTrigger-11987-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3239416 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 3239417 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239417 DEBUG (ScheduledTrigger-11987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3239416 DEBUG (ScheduledTrigger-11987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
[junit4] 2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10023_solr
[junit4] 2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10023_solr]
[junit4] 2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8cf8747a21aafT71cpd1fni40admcqk3y2085e0",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2479979743025839,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479979743025839],
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239418 DEBUG (ScheduledTrigger-11987-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 3239419 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8cf8747a21aafT71cpd1fni40admcqk3y2085e0",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":2479979743025839,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2479979743025839],
[junit4] 2> "_enqueue_time_":2479979754974189,
[junit4] 2> "nodeNames":["127.0.0.1:10023_solr"]}}
[junit4] 2> 3239419 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 2479974568763489 time = 2479979810239539
[junit4] 2> 3239419 INFO (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10023_solr
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 5
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 9
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 3239420 DEBUG (AutoscalingActionExecutor-11988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 64 ms for event id=8cf8747a21aafT71cpd1fni40admcqk3y2085e0
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger2
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - IGNORED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger2
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 3239420 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 3239429 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 3239429 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 3239429 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10023_solr
[junit4] 2> 3239429 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 3239430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3239430 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 3239430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239430 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 3239430 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 3239430 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_162]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_162]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
[junit4] 2> 3239431 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 3239431 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 3239431 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3239438 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 3239438 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
[junit4] 2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr]
[junit4] 2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 3239440 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239460 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239481 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239502 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239522 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3239537 DEBUG (simCloudManagerPool-11989-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239538 DEBUG (simCloudManagerPool-11989-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 3239538 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 3239540 INFO (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 3239540 DEBUG (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239540 DEBUG (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3239541 DEBUG (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 3239541 INFO (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 3239541 DEBUG (simCloudManagerPool-11989-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 3239543 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- deleted /autoscaling/nodeAdded/127.0.0.1:10024_solr
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 15
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10020_solr, 127.0.0.1:10024_solr]
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 3239543 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 3239543 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239543 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239563 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239565 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239565 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239586 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239606 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239606 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239606 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239627 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239627 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239627 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239647 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239647 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239647 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239667 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239667 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239667 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239687 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239687 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239687 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239707 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239708 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239708 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239728 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239748 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239748 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239748 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239768 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239768 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239768 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239789 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239789 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239789 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239809 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239809 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239809 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239829 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239829 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239829 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239850 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239850 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239850 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239870 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239870 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239870 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239890 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239890 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239890 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239910 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239910 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239910 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239930 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239930 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239930 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239951 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239951 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239951 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239971 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239971 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239971 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3239991 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3239991 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3239991 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240011 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240011 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240011 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240031 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240031 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240031 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240052 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240052 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240052 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240072 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240072 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240072 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240092 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240092 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240092 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240112 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240112 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240112 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240133 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240133 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240133 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240160 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240160 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240160 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240180 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240200 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240201 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240201 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240221 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240221 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240221 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240241 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240241 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240241 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240261 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240261 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240261 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240282 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240282 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240282 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240306 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240306 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240306 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240326 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240326 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240326 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240350 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240350 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240350 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240370 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240370 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240370 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240390 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240390 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240390 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240410 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240410 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240410 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240431 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240431 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240431 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240452 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240452 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240452 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240472 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240472 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240473 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240493 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240493 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240493 DEBUG (ScheduledTrigger-11990-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240513 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240513 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240513 DEBUG (ScheduledTrigger-11990-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240533 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240533 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 3240533 DEBUG (ScheduledTrigger-11990-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3240553 DEBUG (ScheduledTrigger-11990-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 3240553 DEBUG (ScheduledTrigger-
[...truncated too long message...]
nit4] 2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"8959e056ec2b2T71cpd1fni40admcqk3y207rzt",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":2416305742201522,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2416305742201522],
[junit4] 2> "nodeNames":["127.0.0.1:10047_solr"]}}
[junit4] 2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 15677 DEBUG (ScheduledTrigger-146-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10047_solr
[junit4] 2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"8959e056ec2b2T71cpd1fni40admcqk3y207rzt",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":2416305742201522,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2416305742201522],
[junit4] 2> "_enqueue_time_":2416307347319672,
[junit4] 2> "nodeNames":["127.0.0.1:10047_solr"]}}
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 15678 DEBUG (AutoscalingActionExecutor-147-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 29 ms for event id=8959e056ec2b2T71cpd1fni40admcqk3y207rzt
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 15678 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 15679 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 15679 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_restore_trigger
[junit4] 2> 15679 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 15679 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F64294E2F6294EBA]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] OK 0.15s J1 | TestTriggerIntegration.testNodeAddedTriggerRestoreState
[junit4] 2> 15680 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 15680 DEBUG (SUITE-TestTriggerIntegration-seed#[F64294E2F6294EBA]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F64294E2F6294EBA-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3004de5d), locale=es-CO, timezone=America/Halifax
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=42242040,total=100663296
[junit4] 2> NOTE: All tests run in this JVM: [TestTriggerIntegration, TestTriggerIntegration, TestTriggerIntegration]
[junit4] Completed [5/5 (2!)] on J1 in 3.58s, 11 tests, 1 failure <<< FAILURES!
[junit4]
[junit4]
[junit4] Tests with failures [seed: F64294E2F6294EBA]:
[junit4] - org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
[junit4] - org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
[junit4]
[junit4]
[junit4] JVM J0: 0.87 .. 13.16 = 12.29s
[junit4] JVM J1: 0.86 .. 16.99 = 16.13s
[junit4] Execution time total: 17 seconds
[junit4] Tests summary: 5 suites, 55 tests, 2 failures
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1568: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1092: There were test failures: 5 suites, 55 tests, 2 failures [seed: F64294E2F6294EBA]
Total time: 18 seconds
[repro] Setting last failure code to 256
[repro] Failures:
[repro] 2/5 failed: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=5b250b4a403f5b572f667d89acab6d07553b2e65, workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #1793
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2