You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/04/10 10:00:19 UTC
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 197 - Still
unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/197/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([1D1E264F0B72E37F:9523AF3031B202D2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState(TestTriggerIntegration.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13044 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_1D1E264F0B72E37F-001/init-core-data-001
[junit4] 2> 825628 INFO (SUITE-TestTriggerIntegration-seed#[1D1E264F0B72E37F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 825630 INFO (SUITE-TestTriggerIntegration-seed#[1D1E264F0B72E37F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 825630 INFO (SUITE-TestTriggerIntegration-seed#[1D1E264F0B72E37F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 825791 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 825791 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 825792 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 825792 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] IGNOR/A 0.06s J1 | TestTriggerIntegration.testCooldown
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testEventFromRestoredState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 825838 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 825840 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 825840 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 826117 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 826117 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 826117 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 826152 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 826152 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 826153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 826153 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 826232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 826232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 826232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 826232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 826252 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 826252 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 826277 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 826277 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 826278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 826278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 826314 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 826314 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 826314 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 826314 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826335 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826716 DEBUG (ScheduledTrigger-634-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826862 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826883 DEBUG (ScheduledTrigger-634-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826883 DEBUG (simCloudManagerPool-633-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 826884 DEBUG (simCloudManagerPool-633-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 826956 DEBUG (ScheduledTrigger-634-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 826956 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 826956 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 826957 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 826992 DEBUG (ScheduledTrigger-634-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 827005 DEBUG (simCloudManagerPool-633-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827005 DEBUG (simCloudManagerPool-633-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 827008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 827008 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 827008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 827009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10002_solr
[junit4] 2> 827009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 827009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 827009 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 827009 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 827028 DEBUG (ScheduledTrigger-634-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 827067 DEBUG (simCloudManagerPool-633-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827067 DEBUG (simCloudManagerPool-633-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 827068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 827068 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 827068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 827068 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 827068 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 827068 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 827068 WARN (ScheduledTrigger-634-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger ran but was already closed
[junit4] 2> 827068 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [5681710672540093], now=5681713628869643
[junit4] 2> 827068 ERROR (ScheduledTrigger-634-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Unexpected exception in NodeAddedTrigger
[junit4] 2> java.lang.RuntimeException: Trigger has been closed
[junit4] 2> at org.apache.solr.cloud.autoscaling.NodeAddedTrigger.run(NodeAddedTrigger.java:121) [java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:589) [java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 827070 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f7c5a5a71bdT628nl585kreui5phdtg4uubfu",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":5681710672540093,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681710672540093],
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 827255 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 827272 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 827276 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 827276 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 827276 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 827276 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 827289 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 827289 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 827289 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 827289 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 827290 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_restore_trigger
[junit4] 2> 827290 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeAddedTriggerRestoreState -Dtests.seed=1D1E264F0B72E37F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=Europe/Sofia -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 1.50s J1 | TestTriggerIntegration.testNodeAddedTriggerRestoreState <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
[junit4] > at __randomizedtesting.SeedInfo.seed([1D1E264F0B72E37F:9523AF3031B202D2]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState(TestTriggerIntegration.java:379)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 827317 DEBUG (AutoscalingActionExecutor-635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f7c5a5a71bdT628nl585kreui5phdtg4uubfu",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":5681710672540093,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681710672540093],
[junit4] 2> "_enqueue_time_":5681723855762143,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 827353 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 827353 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 827359 WARN (AutoscalingActionExecutor-635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.NullPointerException: null
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:320) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 827366 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 827353 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 827374 DEBUG (ScheduledTrigger-634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 827394 DEBUG (AutoscalingActionExecutor-635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 827395 DEBUG (AutoscalingActionExecutor-635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
[junit4] 2> 827395 DEBUG (AutoscalingActionExecutor-635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 3872 ms for event id=142f7c5a5a71bdT628nl585kreui5phdtg4uubfu
[junit4] 2> 827448 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 827448 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 827448 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 827448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 827448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 827448 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 827486 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 827486 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 827486 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 827501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 827501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 827501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 827501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 827557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 827557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 827557 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 827557 DEBUG (ScheduledTrigger-637-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 827610 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 827611 DEBUG (simCloudManagerPool-636-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827612 DEBUG (simCloudManagerPool-636-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 827612 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 827613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 827613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 827615 DEBUG (simCloudManagerPool-636-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827615 DEBUG (simCloudManagerPool-636-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 827615 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 827615 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 827616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 827616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 827616 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 827616 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 827617 DEBUG (simCloudManagerPool-636-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827617 DEBUG (simCloudManagerPool-636-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 827638 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 827638 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 827638 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 827639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 827639 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 827639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 827640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 827646 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 827646 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 827646 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 5681742528646893
[junit4] 2> 827646 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [5681742528646893], now=5681742530669693
[junit4] 2> 827646 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f83c5200eedT628nl585kreui5phdtg4uubfw",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":5681742528646893,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681742528646893],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827647 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 827654 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 827654 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f83c5200eedT628nl585kreui5phdtg4uubfw",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":5681742528646893,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681742528646893],
[junit4] 2> "_enqueue_time_":5681742593939193,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827655 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 827655 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 827655 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 827655 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 827655 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 52 ms for event id=142f83c5200eedT628nl585kreui5phdtg4uubfw
[junit4] 2> 827772 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 5681749017754493
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [5681749017754493], now=5681749019890893
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f8547e7fb7dT628nl585kreui5phdtg4uubfz",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681749017754493,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681749017754493],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827776 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 827793 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 827793 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 827794 DEBUG (ScheduledTrigger-637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 827794 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f8547e7fb7dT628nl585kreui5phdtg4uubfz",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681749017754493,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681749017754493],
[junit4] 2> "_enqueue_time_":5681749035248293,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827794 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 5681742952526093 time = 5681749936144493
[junit4] 2> 827794 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 827808 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 827808 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 827809 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 827809 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 720 ms for event id=142f8547e7fb7dT628nl585kreui5phdtg4uubfz
[junit4] 2> 827810 DEBUG (simCloudManagerPool-636-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827810 DEBUG (simCloudManagerPool-636-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 827811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 827812 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 827812 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 827812 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 827846 DEBUG (simCloudManagerPool-636-thread-9) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 827846 DEBUG (simCloudManagerPool-636-thread-9) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 827846 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 827847 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 827847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 827847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 827849 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 827849 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
[junit4] 2> 827849 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10003_solr]
[junit4] 2> 827849 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f8621775091T628nl585kreui5phdtg4uubg2",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":5681752667803793,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681752667803793],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827849 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 827850 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f8621775091T628nl585kreui5phdtg4uubg2",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":5681752667803793,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681752667803793],
[junit4] 2> "_enqueue_time_":5681752690337493,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827850 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 827851 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 31 ms for event id=142f8621775091T628nl585kreui5phdtg4uubg2
[junit4] 2> 827959 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 827959 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
[junit4] 2> 827959 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10003_solr]
[junit4] 2> 827959 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f876873e959T628nl585kreui5phdtg4uubg5",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":5681758153730393,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681758153730393],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827959 DEBUG (ScheduledTrigger-637-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 827976 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f876873e959T628nl585kreui5phdtg4uubg5",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":5681758153730393,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681758153730393],
[junit4] 2> "_enqueue_time_":5681758174760143,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 827976 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 5681752742305943 time = 5681759015496293
[junit4] 2> 827976 INFO (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
[junit4] 2> 827976 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 827976 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 827976 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 827976 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 827976 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10003_solr
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 5
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 7
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger2
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger2
[junit4] 2> 827977 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeMarkersRegistration
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testListeners
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testSearchRate
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeLostTrigger
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | TestTriggerIntegration.testNodeLostTriggerRestoreState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 828048 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
[junit4] 2> 828050 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 828050 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 828050 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
[junit4] 2> 828050 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 828050 DEBUG (AutoscalingActionExecutor-638-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 3694 ms for event id=142f876873e959T628nl585kreui5phdtg4uubg5
[junit4] 2> 828211 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828212 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
[junit4] 2> 828212 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 828212 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 828212 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 828213 DEBUG (ScheduledTrigger-637-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 828213 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 828213 DEBUG (ScheduledTrigger-637-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 828213 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 828213 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 828214 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10003_solr
[junit4] 2> 828214 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 828214 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 828214 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 828214 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 828214 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 828232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 828232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 828232 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 828261 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 828261 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 828276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
[junit4] 2> 828276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 828276 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 828277 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 828277 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 828277 DEBUG (ScheduledTrigger-640-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828304 DEBUG (ScheduledTrigger-640-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828341 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828377 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828377 DEBUG (simCloudManagerPool-639-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 828378 DEBUG (simCloudManagerPool-639-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 828378 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 828379 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 828379 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 828403 DEBUG (simCloudManagerPool-639-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 828403 DEBUG (simCloudManagerPool-639-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 828403 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 828404 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 828404 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 828404 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 828404 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 828414 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 828414 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 828414 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 828414 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 5681780930016993
[junit4] 2> 828466 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 828466 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 828466 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 828466 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [5681780930016993], now=5681783533726043
[junit4] 2> 828466 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f8cb60676e1T628nl585kreui5phdtg4uubg8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":5681780930016993,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681780930016993],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 828470 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 828523 DEBUG (ScheduledTrigger-640-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10004_solr
[junit4] 2> 828523 DEBUG (AutoscalingActionExecutor-641-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f8cb60676e1T628nl585kreui5phdtg4uubg8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":5681780930016993,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681780930016993],
[junit4] 2> "_enqueue_time_":5681783741718243,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 828524 DEBUG (AutoscalingActionExecutor-641-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 828524 DEBUG (AutoscalingActionExecutor-641-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 828524 DEBUG (AutoscalingActionExecutor-641-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 33 ms for event id=142f8cb60676e1T628nl585kreui5phdtg4uubg8
[junit4] 2> 828548 DEBUG (simCloudManagerPool-639-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 828549 DEBUG (simCloudManagerPool-639-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 828564 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 828564 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 828633 DEBUG (ScheduledTrigger-640-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 828633 DEBUG (ScheduledTrigger-640-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 828633 DEBUG (ScheduledTrigger-640-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 828637 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
[junit4] 2> 828638 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 828638 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 828656 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 828656 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 828656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 828656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 828656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 828656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 828656 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 828656 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:181) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 828657 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 828657 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 828657 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 828685 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 828686 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 828686 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 828722 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 828723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 828723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion -1
[junit4] 2> 828723 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 828743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr]
[junit4] 2> 828743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 828743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
[junit4] 2> 828759 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828797 DEBUG (ScheduledTrigger-643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828832 DEBUG (ScheduledTrigger-643-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828833 DEBUG (simCloudManagerPool-642-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 828833 DEBUG (simCloudManagerPool-642-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 828834 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 828835 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 828835 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 828849 INFO (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 828849 DEBUG (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 828849 DEBUG (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 828849 DEBUG (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 21
[junit4] 2> 828850 INFO (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 828850 DEBUG (simCloudManagerPool-642-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 828850 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 21
[junit4] 2> 828850 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr]
[junit4] 2> 828850 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 828850 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
[junit4] 2> 828852 DEBUG (ScheduledTrigger-643-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 828868 DEBUG (ScheduledTrigger-643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828885 DEBUG (ScheduledTrigger-643-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 828885 DEBUG (ScheduledTrigger-643-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10005_solr at time 5681804489279293
[junit4] 2> 828905 DEBUG (ScheduledTrigger-643-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 828906 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 828906 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 828906 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10005_solr] added at times [5681804489279293], now=5681805542652543
[junit4] 2> 828907 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 828908 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 828958 DEBUG (ScheduledTrigger-643-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10005_solr
[junit4] 2> 828958 DEBUG (AutoscalingActionExecutor-644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "_enqueue_time_":5681805607977843,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 828958 INFO (AutoscalingActionExecutor-644-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "_enqueue_time_":5681805607977843,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 828958 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 828959 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 828959 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 828959 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 828959 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 828959 WARN (AutoscalingActionExecutor-644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:546) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:319) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 828960 DEBUG (AutoscalingActionExecutor-644-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 79 ms for event id=142f923244473dT628nl585kreui5phdtg4uubgb
[junit4] 2> 828960 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 828960 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 828960 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 828978 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 21
[junit4] 2> 828979 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 828979 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 828979 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion -1
[junit4] 2> 828979 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 21
[junit4] 2> 828980 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr]
[junit4] 2> 828980 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr
[junit4] 2> 828994 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr]
[junit4] 2> 828994 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 828995 DEBUG (ScheduledTrigger-646-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 828995 DEBUG (ScheduledTrigger-646-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 828995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
[junit4] 2> 828995 DEBUG (ScheduledTrigger-646-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "_dequeue_time_":5681809979889843,
[junit4] 2> "_enqueue_time_":5681805607977843,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 829017 DEBUG (ScheduledTrigger-646-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 829051 DEBUG (ScheduledTrigger-646-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 829051 DEBUG (ScheduledTrigger-646-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 829052 DEBUG (AutoscalingActionExecutor-647-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "_dequeue_time_":5681809979889843,
[junit4] 2> "_enqueue_time_":5681805607977843,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 829052 INFO (AutoscalingActionExecutor-647-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"142f923244473dT628nl585kreui5phdtg4uubgb",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5681804489279293,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5681804489279293],
[junit4] 2> "_dequeue_time_":5681809979889843,
[junit4] 2> "_enqueue_time_":5681805607977843,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10001_solr
[junit4] 2> 829080 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 829081 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[1D1E264F0B72E37F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 829081 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[
[...truncated too long message...]
y-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1017095066
[copy] Copying 40 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1017095066
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1017095066
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1390607118
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1390607118
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1390607118
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 27ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 502 minutes 24 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 199 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/199/
1 tests failed.
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1523615867055,generation=2,filelist=[_5j6.cfe, _5j6.cfs, _5j6.si, _5k0.doc, _5k0.fdt, _5k0.fdx, _5k0.fnm, _5k0.nvd, _5k0.nvm, _5k0.si, _5k0.tim, _5k0.tip, _5k1.doc, _5k1.fdt, _5k1.fdx, _5k1.fnm, _5k1.nvd, _5k1.nvm, _5k1.si, _5k1.tim, _5k1.tip, _5k3.doc, _5k3.fdt, _5k3.fdx, _5k3.fnm, _5k3.nvd, _5k3.nvm, _5k3.si, _5k3.tim, _5k3.tip, _5k4.doc, _5k4.fdt, _5k4.fdx, _5k4.fnm, _5k4.nvd, _5k4.nvm, _5k4.si, _5k4.tim, _5k4.tip, _5k8.cfe, _5k8.cfs, _5k8.si, _5kf.doc, _5kf.fdt, _5kf.fdx, _5kf.fnm, _5kf.nvd, _5kf.nvm, _5kf.si, _5kf.tim, _5kf.tip, _5kg.doc, _5kg.fdt, _5kg.fdx, _5kg.fnm, _5kg.nvd, _5kg.nvm, _5kg.si, _5kg.tim, _5kg.tip, _5kl.doc, _5kl.fdt, _5kl.fdx, _5kl.fnm, _5kl.nvd, _5kl.nvm, _5kl.si, _5kl.tim, _5kl.tip, _5ko.doc, _5ko.fdt, _5ko.fdx, _5ko.fnm, _5ko.nvd, _5ko.nvm, _5ko.si, _5ko.tim, _5ko.tip, _5kr.doc, _5kr.fdt, _5kr.fdx, _5kr.fnm, _5kr.nvd, _5kr.nvm, _5kr.si, _5kr.tim, _5kr.tip, _5ks.doc, _5ks.fdt, _5ks.fdx, _5ks.fnm, _5ks.nvd, _5ks.nvm, _5ks.si, _5ks.tim, _5ks.tip, _5kt.cfe, _5kt.cfs, _5kt.si, _5ku.doc, _5ku.fdt, _5ku.fdx, _5ku.fnm, _5ku.nvd, _5ku.nvm, _5ku.si, _5ku.tim, _5ku.tip, _5kv.doc, _5kv.fdt, _5kv.fdx, _5kv.fnm, _5kv.nvd, _5kv.nvm, _5kv.si, _5kv.tim, _5kv.tip, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_2]}]> but was:<[{indexVersion=1523615867055,generation=2,filelist=[_5j6.cfe, _5j6.cfs, _5j6.si, _5k0.doc, _5k0.fdt, _5k0.fdx, _5k0.fnm, _5k0.nvd, _5k0.nvm, _5k0.si, _5k0.tim, _5k0.tip, _5k1.doc, _5k1.fdt, _5k1.fdx, _5k1.fnm, _5k1.nvd, _5k1.nvm, _5k1.si, _5k1.tim, _5k1.tip, _5k3.doc, _5k3.fdt, _5k3.fdx, _5k3.fnm, _5k3.nvd, _5k3.nvm, _5k3.si, _5k3.tim, _5k3.tip, _5k4.doc, _5k4.fdt, _5k4.fdx, _5k4.fnm, _5k4.nvd, _5k4.nvm, _5k4.si, _5k4.tim, _5k4.tip, _5k8.cfe, _5k8.cfs, _5k8.si, _5kf.doc, _5kf.fdt, _5kf.fdx, _5kf.fnm, _5kf.nvd, _5kf.nvm, _5kf.si, _5kf.tim, _5kf.tip, _5kg.doc, _5kg.fdt, _5kg.fdx, _5kg.fnm, _5kg.nvd, _5kg.nvm, _5kg.si, _5kg.tim, _5kg.tip, _5kl.doc, _5kl.fdt, _5kl.fdx, _5kl.fnm, _5kl.nvd, _5kl.nvm, _5kl.si, _5kl.tim, _5kl.tip, _5ko.doc, _5ko.fdt, _5ko.fdx, _5ko.fnm, _5ko.nvd, _5ko.nvm, _5ko.si, _5ko.tim, _5ko.tip, _5kr.doc, _5kr.fdt, _5kr.fdx, _5kr.fnm, _5kr.nvd, _5kr.nvm, _5kr.si, _5kr.tim, _5kr.tip, _5ks.doc, _5ks.fdt, _5ks.fdx, _5ks.fnm, _5ks.nvd, _5ks.nvm, _5ks.si, _5ks.tim, _5ks.tip, _5kt.cfe, _5kt.cfs, _5kt.si, _5ku.doc, _5ku.fdt, _5ku.fdx, _5ku.fnm, _5ku.nvd, _5ku.nvm, _5ku.si, _5ku.tim, _5ku.tip, _5kv.doc, _5kv.fdt, _5kv.fdx, _5kv.fnm, _5kv.nvd, _5kv.nvm, _5kv.si, _5kv.tim, _5kv.tip, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_2]}, {indexVersion=1523615867055,generation=3,filelist=[_5kw.cfe, _5kw.cfs, _5kw.si, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1523615867055,generation=2,filelist=[_5j6.cfe, _5j6.cfs, _5j6.si, _5k0.doc, _5k0.fdt, _5k0.fdx, _5k0.fnm, _5k0.nvd, _5k0.nvm, _5k0.si, _5k0.tim, _5k0.tip, _5k1.doc, _5k1.fdt, _5k1.fdx, _5k1.fnm, _5k1.nvd, _5k1.nvm, _5k1.si, _5k1.tim, _5k1.tip, _5k3.doc, _5k3.fdt, _5k3.fdx, _5k3.fnm, _5k3.nvd, _5k3.nvm, _5k3.si, _5k3.tim, _5k3.tip, _5k4.doc, _5k4.fdt, _5k4.fdx, _5k4.fnm, _5k4.nvd, _5k4.nvm, _5k4.si, _5k4.tim, _5k4.tip, _5k8.cfe, _5k8.cfs, _5k8.si, _5kf.doc, _5kf.fdt, _5kf.fdx, _5kf.fnm, _5kf.nvd, _5kf.nvm, _5kf.si, _5kf.tim, _5kf.tip, _5kg.doc, _5kg.fdt, _5kg.fdx, _5kg.fnm, _5kg.nvd, _5kg.nvm, _5kg.si, _5kg.tim, _5kg.tip, _5kl.doc, _5kl.fdt, _5kl.fdx, _5kl.fnm, _5kl.nvd, _5kl.nvm, _5kl.si, _5kl.tim, _5kl.tip, _5ko.doc, _5ko.fdt, _5ko.fdx, _5ko.fnm, _5ko.nvd, _5ko.nvm, _5ko.si, _5ko.tim, _5ko.tip, _5kr.doc, _5kr.fdt, _5kr.fdx, _5kr.fnm, _5kr.nvd, _5kr.nvm, _5kr.si, _5kr.tim, _5kr.tip, _5ks.doc, _5ks.fdt, _5ks.fdx, _5ks.fnm, _5ks.nvd, _5ks.nvm, _5ks.si, _5ks.tim, _5ks.tip, _5kt.cfe, _5kt.cfs, _5kt.si, _5ku.doc, _5ku.fdt, _5ku.fdx, _5ku.fnm, _5ku.nvd, _5ku.nvm, _5ku.si, _5ku.tim, _5ku.tip, _5kv.doc, _5kv.fdt, _5kv.fdx, _5kv.fnm, _5kv.nvd, _5kv.nvm, _5kv.si, _5kv.tim, _5kv.tip, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_2]}]> but was:<[{indexVersion=1523615867055,generation=2,filelist=[_5j6.cfe, _5j6.cfs, _5j6.si, _5k0.doc, _5k0.fdt, _5k0.fdx, _5k0.fnm, _5k0.nvd, _5k0.nvm, _5k0.si, _5k0.tim, _5k0.tip, _5k1.doc, _5k1.fdt, _5k1.fdx, _5k1.fnm, _5k1.nvd, _5k1.nvm, _5k1.si, _5k1.tim, _5k1.tip, _5k3.doc, _5k3.fdt, _5k3.fdx, _5k3.fnm, _5k3.nvd, _5k3.nvm, _5k3.si, _5k3.tim, _5k3.tip, _5k4.doc, _5k4.fdt, _5k4.fdx, _5k4.fnm, _5k4.nvd, _5k4.nvm, _5k4.si, _5k4.tim, _5k4.tip, _5k8.cfe, _5k8.cfs, _5k8.si, _5kf.doc, _5kf.fdt, _5kf.fdx, _5kf.fnm, _5kf.nvd, _5kf.nvm, _5kf.si, _5kf.tim, _5kf.tip, _5kg.doc, _5kg.fdt, _5kg.fdx, _5kg.fnm, _5kg.nvd, _5kg.nvm, _5kg.si, _5kg.tim, _5kg.tip, _5kl.doc, _5kl.fdt, _5kl.fdx, _5kl.fnm, _5kl.nvd, _5kl.nvm, _5kl.si, _5kl.tim, _5kl.tip, _5ko.doc, _5ko.fdt, _5ko.fdx, _5ko.fnm, _5ko.nvd, _5ko.nvm, _5ko.si, _5ko.tim, _5ko.tip, _5kr.doc, _5kr.fdt, _5kr.fdx, _5kr.fnm, _5kr.nvd, _5kr.nvm, _5kr.si, _5kr.tim, _5kr.tip, _5ks.doc, _5ks.fdt, _5ks.fdx, _5ks.fnm, _5ks.nvd, _5ks.nvm, _5ks.si, _5ks.tim, _5ks.tip, _5kt.cfe, _5kt.cfs, _5kt.si, _5ku.doc, _5ku.fdt, _5ku.fdx, _5ku.fnm, _5ku.nvd, _5ku.nvm, _5ku.si, _5ku.tim, _5ku.tip, _5kv.doc, _5kv.fdt, _5kv.fdx, _5kv.fnm, _5kv.nvd, _5kv.nvm, _5kv.si, _5kv.tim, _5kv.tip, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_2]}, {indexVersion=1523615867055,generation=3,filelist=[_5kw.cfe, _5kw.cfs, _5kw.si, _5m9.doc, _5m9.fdt, _5m9.fdx, _5m9.fnm, _5m9.nvd, _5m9.nvm, _5m9.si, _5m9.tim, _5m9.tip, _5mb.cfe, _5mb.cfs, _5mb.si, _5mc.doc, _5mc.fdt, _5mc.fdx, _5mc.fnm, _5mc.nvd, _5mc.nvm, _5mc.si, _5mc.tim, _5mc.tip, _5md.doc, _5md.fdt, _5md.fdx, _5md.fnm, _5md.nvd, _5md.nvm, _5md.si, _5md.tim, _5md.tip, _5me.doc, _5me.fdt, _5me.fdx, _5me.fnm, _5me.nvd, _5me.nvm, _5me.si, _5me.tim, _5me.tip, _5mf.doc, _5mf.fdt, _5mf.fdx, _5mf.fnm, _5mf.nvd, _5mf.nvm, _5mf.si, _5mf.tim, _5mf.tip, _5mg.doc, _5mg.fdt, _5mg.fdx, _5mg.fnm, _5mg.nvd, _5mg.nvm, _5mg.si, _5mg.tim, _5mg.tip, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([DAFFDC8E84BA5E7B:FF28C7BEF4F25078]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14289 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/init-core-data-001
[junit4] 2> 4076898 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4076899 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4076899 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4076899 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 4076899 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e0fa773{/solr,null,AVAILABLE}
[junit4] 2> 4076900 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@46b6d908{HTTP/1.1,[http/1.1]}{127.0.0.1:46040}
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server Started @4077041ms
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=46040}
[junit4] 2> 4076901 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-13T10:34:13.147Z
[junit4] 2> 4076901 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/solr.xml
[junit4] 2> 4076925 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4076925 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4076926 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4077041 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077062 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077062 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077064 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/.
[junit4] 2> 4077064 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4077065 INFO (coreLoadExecutor-4717-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4077111 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 4077139 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4077141 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 4077170 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 4077171 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077171 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4077171 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/./collection1/data/]
[junit4] 2> 4077173 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=90.7744140625, floorSegmentMB=1.939453125, forceMergeDeletesPctAllowed=16.205093016958976, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5398778681375698
[junit4] 2> 4077314 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4077314 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4077314 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@61de7b48
[junit4] 2> 4077315 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@20b0938[collection1] main]
[junit4] 2> 4077315 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1/conf
[junit4] 2> 4077316 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 4077316 INFO (coreLoadExecutor-4717-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4077316 INFO (searcherExecutor-4718-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@20b0938[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4077320 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4077321 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4077321 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4077321 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 4077322 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a9a7840{/solr,null,AVAILABLE}
[junit4] 2> 4077322 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5d7fcdab{HTTP/1.1,[http/1.1]}{127.0.0.1:46766}
[junit4] 2> 4077322 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server Started @4077463ms
[junit4] 2> 4077322 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=46766}
[junit4] 2> 4077322 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4077322 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4077323 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 4077323 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 4077323 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4077323 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-13T10:34:13.569Z
[junit4] 2> 4077323 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/solr.xml
[junit4] 2> 4077343 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4077343 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4077344 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4077448 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077474 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077474 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077477 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/.
[junit4] 2> 4077477 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4077477 INFO (coreLoadExecutor-4727-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4077548 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 4077591 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4077592 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 4077592 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 4077593 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4077593 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4077593 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/./collection1/data/]
[junit4] 2> 4077595 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=90.7744140625, floorSegmentMB=1.939453125, forceMergeDeletesPctAllowed=16.205093016958976, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5398778681375698
[junit4] 2> 4077710 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4077710 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4077719 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@69cccdf2
[junit4] 2> 4077720 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@450ebbf7[collection1] main]
[junit4] 2> 4077720 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-002/collection1/conf
[junit4] 2> 4077722 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 4077722 INFO (coreLoadExecutor-4727-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4077722 INFO (searcherExecutor-4728-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@450ebbf7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4077725 INFO (qtp1478977714-65530) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 4077725 INFO (indexFetcher-4732-thread-1) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3de5bfe0[collection1] main]
[junit4] 2> 4077726 INFO (searcherExecutor-4728-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3de5bfe0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4078240 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5d7fcdab{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4078241 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=150096332
[junit4] 2> 4078241 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4078241 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1910bac: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7491b704
[junit4] 2> 4078246 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4078246 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@68bb3da9: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@e1ceb41
[junit4] 2> 4078258 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4078258 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@69450003: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@f60b336
[junit4] 2> 4078271 INFO (coreCloseExecutor-4733-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@334cfeea
[junit4] 2> 4078271 INFO (coreCloseExecutor-4733-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=334cfeea
[junit4] 2> 4078272 INFO (coreCloseExecutor-4733-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@17a607eb: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@ea7ecd2
[junit4] 2> 4078297 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2a9a7840{/solr,null,UNAVAILABLE}
[junit4] 2> 4078297 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4078300 INFO (qtp1478977714-65536) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 4078300 INFO (qtp1478977714-65536) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 4078312 INFO (qtp1478977714-65535) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 4078312 INFO (qtp1478977714-65535) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1deb5979 commitCommandVersion:0
[junit4] 2> 4078313 INFO (qtp1478977714-65535) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5211be8f[collection1] main]
[junit4] 2> 4078314 INFO (qtp1478977714-65535) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 4078314 INFO (searcherExecutor-4718-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5211be8f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4078314 INFO (qtp1478977714-65535) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
[junit4] 2> 4078316 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@46b6d908{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4078316 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1047449243
[junit4] 2> 4078317 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4078317 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@642d9732: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1e33951f
[junit4] 2> 4078318 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4078318 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@59dffe37: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@e1ceb41
[junit4] 2> 4078319 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4078319 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@69613d9b: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@f60b336
[junit4] 2> 4078328 INFO (coreCloseExecutor-4735-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@100d4370
[junit4] 2> 4078329 INFO (coreCloseExecutor-4735-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=100d4370
[junit4] 2> 4078329 INFO (coreCloseExecutor-4735-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a4799d2: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@67bd21d8
[junit4] 2> 4078331 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5e0fa773{/solr,null,UNAVAILABLE}
[junit4] 2> 4078331 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4078353 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4078354 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4078354 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4078365 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 4078381 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ada5734{/solr,null,AVAILABLE}
[junit4] 2> 4078382 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@382b3c78{HTTP/1.1,[http/1.1]}{127.0.0.1:37742}
[junit4] 2> 4078382 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.e.j.s.Server Started @4078523ms
[junit4] 2> 4078382 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=37742}
[junit4] 2> 4078382 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4078382 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4078430 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.4.0
[junit4] 2> 4078430 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 4078430 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4078430 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-13T10:34:14.676Z
[junit4] 2> 4078431 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/solr.xml
[junit4] 2> 4078435 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4078435 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4078436 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4078526 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4078542 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4078542 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4078552 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/.
[junit4] 2> 4078552 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[DAFFDC8E84BA5E7B]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4078553 INFO (coreLoadExecutor-4742-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4078575 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 4078603 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4078604 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 4078604 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 4078605 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42de8a09
[junit4] 2> 4078605 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4078605 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/./collection1/data/]
[junit4] 2> 4078607 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4300cf80
[junit4] 2> 4078736 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4078736 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4078737 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=8, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26060502327489976]
[junit4] 2> 4078738 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9e49334[collection1] main]
[junit4] 2> 4078738 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_DAFFDC8E84BA5E7B-001/solr-instance-001/collection1/conf
[junit4] 2> 4078739 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 4078739 INFO (coreLoadExecutor-4742-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4078739 INFO (searcherExecutor-4743-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9e49334[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4078752 INFO (qtp943432979-65572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
[junit4] 2> 4078753 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 4078768 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 4078771 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
[junit4] 2> 4078772 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 4078772 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
[junit4] 2> 4078772 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
[junit4] 2> 4078773 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
[junit4] 2> 4078773 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
[junit4] 2> 4078773 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
[junit4] 2> 4078774 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
[junit4] 2> 4078774 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
[junit4] 2> 4078774 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
[junit4] 2> 4078775 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
[junit4] 2> 4078775 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
[junit4] 2> 4078788 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
[junit4] 2> 4078788 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
[junit4] 2> 4078788 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
[junit4] 2> 4078789 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
[junit4] 2> 4078789 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
[junit4] 2> 4078789 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
[junit4] 2> 4078790 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
[junit4] 2> 4078790 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
[junit4] 2> 4078790 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
[junit4] 2> 4078791 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
[junit4] 2> 4078791 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
[junit4] 2> 4078791 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
[junit4] 2> 4078792 INFO (qtp943432979-65576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
[junit4] 2> 4078792 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
[junit4] 2> 4078792 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
[junit4] 2> 4078793 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
[junit4] 2> 4078793 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
[junit4] 2> 4078794 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
[junit4] 2> 4078796 INFO (qtp943432979-65577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 1
[junit4] 2> 4078833 INFO (qtp943432979-65580) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 29
[junit4] 2> 4078834 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
[junit4] 2> 4078834 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
[junit4] 2> 4078835 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
[junit4] 2> 4078835 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
[junit4] 2> 4078835 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
[junit4] 2> 4078836 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
[junit4] 2> 4078836 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
[junit4] 2> 4078836 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
[junit4] 2> 4078837 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
[junit4] 2> 4078837 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
[junit4] 2> 4078837 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
[junit4] 2> 4078838 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
[junit4] 2> 4078838 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
[junit4] 2> 4078838 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
[junit4] 2> 4078838 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
[junit4] 2> 4078839 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
[junit4] 2> 4078839 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
[junit4] 2> 4078839 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
[junit4] 2> 4078840 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
[junit4] 2> 4078840 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
[junit4] 2> 4078861 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
[junit4] 2> 4078861 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
[junit4] 2> 4078861 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
[junit4] 2> 4078862 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
[junit4] 2> 4078862 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
[junit4] 2> 4078862 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
[junit4] 2> 4078863 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
[junit4] 2> 4078863 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
[junit4] 2> 4078863 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
[junit4] 2> 4078864 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
[junit4] 2> 4078864 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
[junit4] 2> 4078864 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
[junit4] 2> 4078877 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 13
[junit4] 2> 4078878 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
[junit4] 2> 4078878 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
[junit4] 2> 4078878 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
[junit4] 2> 4078879 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
[junit4] 2> 4078879 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
[junit4] 2> 4078879 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
[junit4] 2> 4078880 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
[junit4] 2> 4078880 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
[junit4] 2> 4078880 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
[junit4] 2> 4078881 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
[junit4] 2> 4078897 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
[junit4] 2> 4078898 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
[junit4] 2> 4078898 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
[junit4] 2> 4078898 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
[junit4] 2> 4078899 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
[junit4] 2> 4078899 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
[junit4] 2> 4078899 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
[junit4] 2> 4078900 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
[junit4] 2> 4078900 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
[junit4] 2> 4078900 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
[junit4] 2> 4078901 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
[junit4] 2> 4078913 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 12
[junit4] 2> 4078914 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
[junit4] 2> 4078914 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
[junit4] 2> 4078914 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
[junit4] 2> 4078914 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
[junit4] 2> 4078915 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
[junit4] 2> 4078915 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
[junit4] 2> 4078915 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
[junit4] 2> 4078916 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
[junit4] 2> 4078916 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
[junit4] 2> 4078916 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
[junit4] 2> 4078917 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
[junit4] 2> 4078934 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 17
[junit4] 2> 4078934 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
[junit4] 2> 4078935 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
[junit4] 2> 4078935 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
[junit4] 2> 4078935 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
[junit4] 2> 4078936 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
[junit4] 2> 4078936 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
[junit4] 2> 4078936 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
[junit4] 2> 4078936 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
[junit4] 2> 4078937 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
[junit4] 2> 4078937 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
[junit4] 2> 4078974 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 37
[junit4] 2> 4078975 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
[junit4] 2> 4078975 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
[junit4] 2> 4078975 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
[junit4] 2> 4078976 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
[junit4] 2> 4078976 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
[junit4] 2> 4078976 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
[junit4] 2> 4078976 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
[junit4] 2> 4078995 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 17
[junit4] 2> 4078995 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
[junit4] 2> 4078995 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
[junit4] 2> 4078996 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
[junit4] 2> 4078996 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
[junit4] 2> 4078996 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
[junit4] 2> 4078997 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
[junit4] 2> 4078997 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
[junit4] 2> 4078997 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
[junit4] 2> 4078998 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
[junit4] 2> 4078998 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
[junit4] 2> 4078998 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
[junit4] 2> 4079031 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 32
[junit4] 2> 4079032 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
[junit4] 2> 4079032 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
[junit4] 2> 4079033 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
[junit4] 2> 4079033 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
[junit4] 2> 4079034 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
[junit4] 2> 4079034 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
[junit4] 2> 4079034 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
[junit4] 2> 4079035 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
[junit4] 2> 4079035 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
[junit4] 2> 4079047 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
[junit4] 2> 4079048 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
[junit4] 2> 4079051 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
[junit4] 2> 4079068 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
[junit4] 2> 4079068 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
[junit4] 2> 4079069 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
[junit4] 2> 4079069 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
[junit4] 2> 4079069 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
[junit4] 2> 4079070 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
[junit4] 2> 4079070 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
[junit4] 2> 4079070 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
[junit4] 2> 4079070 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
[junit4] 2> 4079071 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
[junit4] 2> 4079071 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
[junit4] 2> 4079071 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
[junit4] 2> 4079072 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
[junit4] 2> 4079084 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
[junit4] 2> 4079085 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
[junit4] 2> 4079085 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
[junit4] 2> 4079085 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
[junit4] 2> 4079086 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
[junit4] 2> 4079086 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
[junit4] 2> 4079086 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
[junit4] 2> 4079087 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
[junit4] 2> 4079087 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
[junit4] 2> 4079087 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
[junit4] 2> 4079088 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
[junit4] 2> 4079105 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 17
[junit4] 2> 4079105 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
[junit4] 2> 4079106 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
[junit4] 2> 4079106 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
[junit4] 2> 4079106 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
[junit4] 2> 4079107 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
[junit4] 2> 4079107 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
[junit4] 2> 4079107 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
[junit4] 2> 4079108 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
[junit4] 2> 4079108 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
[junit4] 2> 4079108 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
[junit4] 2> 4079121 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
[junit4] 2> 4079121 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
[junit4] 2> 4079121 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
[junit4] 2> 4079122 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
[junit4] 2> 4079122 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
[junit4] 2> 4079122 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
[junit4] 2> 4079122 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
[junit4] 2> 4079123 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
[junit4] 2> 4079123 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
[junit4] 2> 4079123 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
[junit4] 2> 4079124 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
[junit4] 2> 4079124 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
[junit4] 2> 4079124 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
[junit4] 2> 4079124 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
[junit4] 2> 4079141 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
[junit4] 2> 4079142 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
[junit4] 2> 4079142 INFO (qtp943432979-65578) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
[junit4] 2> 4079142 INFO (qtp943432979-65583) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
[ju
[...truncated too long message...]
y-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1567657602
[copy] Copying 40 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1567657602
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null1567657602
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1964697925
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1964697925
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1964697925
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 26ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 481 minutes 14 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 198 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/198/
3 tests failed.
FAILED: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test
Error Message:
Error from server at https://127.0.0.1:40759/solr: Could not restore core
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:40759/solr: Could not restore core
at __randomizedtesting.SeedInfo.seed([93925E63CC51A4B4:1BC661B962ADC94C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:287)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([93925E63CC51A4B4:B5F8E05AD75E0BD8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger(TestTriggerIntegration.java:408)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93925E63CC51A4B4:A02076A7C1E67E03]:0)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:424)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:129)
at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:113)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:113)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:197)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4628)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4144)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2335)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3570)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3522)
at org.apache.lucene.index.RandomIndexWriter.maybeFlushOrCommit(RandomIndexWriter.java:216)
at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:188)
at org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit(TestDocTermOrds.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
Build Log:
[...truncated 12986 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_93925E63CC51A4B4-001/init-core-data-001
[junit4] 2> 172481 WARN (SUITE-TestTriggerIntegration-seed#[93925E63CC51A4B4]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 172482 INFO (SUITE-TestTriggerIntegration-seed#[93925E63CC51A4B4]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 172516 INFO (SUITE-TestTriggerIntegration-seed#[93925E63CC51A4B4]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 172533 INFO (SUITE-TestTriggerIntegration-seed#[93925E63CC51A4B4]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 172791 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 172861 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 172863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 172863 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] IGNOR/A 0.08s J2 | TestTriggerIntegration.testSearchRate
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 172970 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 173009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 173009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 173043 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 173043 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 173066 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 173067 DEBUG (ScheduledTrigger-107-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173079 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 173174 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 173174 DEBUG (ScheduledTrigger-107-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173175 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 173175 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 173175 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 173189 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 173189 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 173202 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 173202 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 173203 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 173203 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 173222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 173223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 173223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 173223 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173294 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173332 DEBUG (ScheduledTrigger-110-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173384 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173409 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173444 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173482 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173517 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173554 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173590 DEBUG (simCloudManagerPool-109-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 173590 DEBUG (simCloudManagerPool-109-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 173590 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173605 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 173605 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 173605 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 173643 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173663 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173700 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 173725 DEBUG (simCloudManagerPool-109-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 173725 DEBUG (simCloudManagerPool-109-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 173733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 173733 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 173733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 173733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 173734 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 173737 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 173737 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 173737 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 173737 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 5807910489118412
[junit4] 2> 173773 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 173773 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 173773 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 173774 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [5807910489118412], now=5807912291932112
[junit4] 2> 173776 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a243899756ccTb6dcrwj17bt0ghdhsu6uyamuw",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":5807910489118412,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5807910489118412],
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 174066 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 174191 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 174192 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 174193 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 174193 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 174193 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 174193 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 174194 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 174194 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 174194 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 174194 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 174214 DEBUG (ScheduledTrigger-110-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
[junit4] 2> 174227 DEBUG (AutoscalingActionExecutor-111-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a243899756ccTb6dcrwj17bt0ghdhsu6uyamuw",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":5807910489118412,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5807910489118412],
[junit4] 2> "_enqueue_time_":5807926959279312,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 174250 DEBUG (AutoscalingActionExecutor-111-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 174250 DEBUG (AutoscalingActionExecutor-111-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 174250 DEBUG (AutoscalingActionExecutor-111-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1777 ms for event id=14a243899756ccTb6dcrwj17bt0ghdhsu6uyamuw
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeAddedTrigger -Dtests.seed=93925E63CC51A4B4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ca-ES -Dtests.timezone=Pacific/Samoa -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.33s J2 | TestTriggerIntegration.testNodeAddedTrigger <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
[junit4] > at __randomizedtesting.SeedInfo.seed([93925E63CC51A4B4:B5F8E05AD75E0BD8]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger(TestTriggerIntegration.java:408)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testCooldown
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testNodeLostTrigger
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 174394 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174520 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174540 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174576 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174812 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174813 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 174813 DEBUG (ScheduledTrigger-110-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 174885 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174885 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 174885 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 174906 DEBUG (ScheduledTrigger-110-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174907 DEBUG (ScheduledTrigger-110-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 174907 DEBUG (ScheduledTrigger-110-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 174994 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 174994 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 174995 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 175141 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 175141 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 175141 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 175562 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 175563 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 175563 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 175598 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 175599 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 175599 DEBUG (ScheduledTrigger-110-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 175600 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 175600 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 175601 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 175601 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 175601 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 175601 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 175601 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 175602 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 175602 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 175602 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 175673 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 175673 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 175711 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 175711 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 175727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 175753 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 175753 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 175893 DEBUG (ScheduledTrigger-113-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 175907 DEBUG (simCloudManagerPool-112-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 175908 DEBUG (simCloudManagerPool-112-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 176192 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 176193 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 176227 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 176228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 176228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 176228 DEBUG (simCloudManagerPool-112-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 176244 DEBUG (simCloudManagerPool-112-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 176245 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 176245 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 176245 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 176245 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 176245 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 176245 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 176245 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 176264 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 176266 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 176266 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 176300 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 176300 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 176300 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 176337 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 176337 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 176337 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 176338 DEBUG (simCloudManagerPool-112-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 176338 DEBUG (simCloudManagerPool-112-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 176339 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 176374 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 176374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 176374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 176374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10003_solr
[junit4] 2> 176374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 176374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 176374 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 176374 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 176374 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 176374 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [5808042313008762], now=5808042332719662
[junit4] 2> 176375 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a2623ae8467aTb6dcrwj17bt0ghdhsu6uyamuz",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":5808042313008762,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808042313008762],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 176376 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 176446 DEBUG (ScheduledTrigger-113-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
[junit4] 2> 176447 DEBUG (AutoscalingActionExecutor-114-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a2623ae8467aTb6dcrwj17bt0ghdhsu6uyamuz",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":5808042313008762,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808042313008762],
[junit4] 2> "_enqueue_time_":5808042440598362,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 176447 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 3
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 176448 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 176449 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_restore_trigger
[junit4] 2> 176449 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testListeners
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testEventFromRestoredState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 176539 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 176539 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 176556 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 176556 DEBUG (AutoscalingActionExecutor-114-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 176556 DEBUG (AutoscalingActionExecutor-114-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
[junit4] 2> 176556 DEBUG (AutoscalingActionExecutor-114-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 5470 ms for event id=14a2623ae8467aTb6dcrwj17bt0ghdhsu6uyamuz
[junit4] 2> 176609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 176610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 176610 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 176610 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 176610 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 176610 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 176610 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 176611 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 176611 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 176611 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 176612 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 176667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 176667 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 176668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
[junit4] 2> 176668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 176718 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 176755 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 176755 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 176755 DEBUG (ScheduledTrigger-116-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 176882 DEBUG (simCloudManagerPool-115-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 176883 DEBUG (simCloudManagerPool-115-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 176942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 176942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 176942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 176964 INFO (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 176964 DEBUG (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 176965 DEBUG (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 176965 DEBUG (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 177066 INFO (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 177066 DEBUG (simCloudManagerPool-115-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 177066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 177067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 177067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 177067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 177080 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 177080 DEBUG (ScheduledTrigger-116-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 177081 DEBUG (ScheduledTrigger-116-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 177081 DEBUG (ScheduledTrigger-116-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 5808077654319262
[junit4] 2> 177101 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 177153 DEBUG (ScheduledTrigger-116-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 177154 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 177154 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 177154 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [5808077654319262], now=5808081303618062
[junit4] 2> 177154 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 177154 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 177210 DEBUG (ScheduledTrigger-116-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10004_solr
[junit4] 2> 177211 DEBUG (AutoscalingActionExecutor-117-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "_enqueue_time_":5808081326400762,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 177211 INFO (AutoscalingActionExecutor-117-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "_enqueue_time_":5808081326400762,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 177226 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 177226 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 177226 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 177226 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 177226 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 177227 WARN (AutoscalingActionExecutor-117-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:546) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:319) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 177227 DEBUG (AutoscalingActionExecutor-117-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 798 ms for event id=14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2
[junit4] 2> 177227 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 177227 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 177227 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 177229 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 177283 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 177283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 177283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion -1
[junit4] 2> 177283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 177283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 177284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10000_solr
[junit4] 2> 177335 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 177335 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 177355 DEBUG (ScheduledTrigger-119-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 177356 DEBUG (ScheduledTrigger-119-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
[junit4] 2> 177371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 177372 DEBUG (ScheduledTrigger-119-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "_dequeue_time_":5808092202008612,
[junit4] 2> "_enqueue_time_":5808081326400762,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 177373 DEBUG (ScheduledTrigger-119-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 177445 DEBUG (ScheduledTrigger-119-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 177445 DEBUG (ScheduledTrigger-119-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 177495 DEBUG (AutoscalingActionExecutor-120-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "_dequeue_time_":5808092202008612,
[junit4] 2> "_enqueue_time_":5808081326400762,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 177496 INFO (AutoscalingActionExecutor-120-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808077654319262,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808077654319262],
[junit4] 2> "_dequeue_time_":5808092202008612,
[junit4] 2> "_enqueue_time_":5808081326400762,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 177499 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10000_solr
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 177500 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 177526 DEBUG (AutoscalingActionExecutor-120-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 177542 DEBUG (AutoscalingActionExecutor-120-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 177542 DEBUG (AutoscalingActionExecutor-120-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2335 ms for event id=14a26a7569e09eTb6dcrwj17bt0ghdhsu6uyamv2
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testNodeMarkersRegistration
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J2 | TestTriggerIntegration.testNodeLostTriggerRestoreState
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> 177656 DEBUG (ScheduledTrigger-119-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 177884 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 177884 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 177884 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10000_solr
[junit4] 2> 177884 INFO (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 177884 DEBUG (ScheduledTrigger-119-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 177884 DEBUG (ScheduledTrigger-119-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 177885 DEBUG (ScheduledTrigger-119-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 177885 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 177885 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 177886 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 177891 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 177931 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 177932 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 177932 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[93925E63CC51A4B4]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 177987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 177987 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 178080 DEBUG (simCloudManagerPool-121-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 178094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion -1
[junit4] 2> 178094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 178094 DEBUG (simCloudManagerPool-121-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 178117 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 178130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 178130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 15
[junit4] 2> 178130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 178132 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 178132 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 178132 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178133 DEBUG (simCloudManagerPool-121-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 178133 DEBUG (simCloudManagerPool-121-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 178135 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 178135 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 178135 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 178136 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 178136 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178136 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178136 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 178152 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178152 DEBUG (simCloudManagerPool-121-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 178153 DEBUG (simCloudManagerPool-121-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 178188 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178188 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178188 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178209 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178209 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178210 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178210 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 178211 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 178211 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 178211 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 178211 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 178211 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 178211 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178236 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178236 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 178236 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10005_solr at time 5808135434972662
[junit4] 2> 178236 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10005_solr] added at times [5808135434972662], now=5808135437518362
[junit4] 2> 178237 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a277e968adf6Tb6dcrwj17bt0ghdhsu6uyamv6",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808135434972662,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808135434972662],
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 178237 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 178238 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10005_solr
[junit4] 2> 178238 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a277e968adf6Tb6dcrwj17bt0ghdhsu6uyamv6",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":5808135434972662,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808135434972662],
[junit4] 2> "_enqueue_time_":5808135484706712,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 178238 INFO (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
[junit4] 2> 178254 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 178254 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 178254 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 178254 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 783 ms for event id=14a277e968adf6Tb6dcrwj17bt0ghdhsu6uyamv6
[junit4] 2> 178362 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 178362 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178362 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 178384 DEBUG (ScheduledTrigger-122-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 178384 DEBUG (ScheduledTrigger-122-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178384 DEBUG (ScheduledTrigger-122-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 178398 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 178398 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 178399 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10005_solr at time 5808143544140212
[junit4] 2> 178399 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10005_solr] added at times [5808143544140212], now=5808143549185512
[junit4] 2> 178399 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"14a279ccc0c1b4Tb6dcrwj17bt0ghdhsu6uyamv9",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":5808143544140212,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808143544140212],
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 178399 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
[junit4] 2> 178400 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10005_solr
[junit4] 2> 178401 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"14a279ccc0c1b4Tb6dcrwj17bt0ghdhsu6uyamv9",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":5808143544140212,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[5808143544140212],
[junit4] 2> "_enqueue_time_":5808143567743712,
[junit4] 2> "nodeNames":["127.0.0.1:10005_solr"]}}
[junit4] 2> 178401 INFO (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration last action at 5808135539962012 time = 5808143655653162
[junit4] 2> 178401 INFO (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
[junit4] 2> 178419 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 178419 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
[junit4] 2> 178419 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 178419 DEBUG (AutoscalingActionExecutor-123-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 913 ms for event id=14a279ccc0c1b4Tb6dcrwj17bt0ghdhsu6uyamv9
[junit4] 2> 178420 DEBUG (simCloudManagerPool-121-thread-8) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 178420 DEBUG (simCloudManagerPool-121-thread-8) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 178421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 178421 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 178421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr, 127.0.0.1:10002_solr]
[junit4] 2> 178421 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
[junit4] 2> 178422 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 178455 DEBUG (ScheduledTrigger-122-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 178456 DEBUG (simCloudManagerPool-121-thread-9) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 178456 DEBUG (simCloudManagerPool-121-thread-9) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 178457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 178457 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 178457 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr, 127.0.0.1:10002_solr]
[junit4] 2> 178458 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 178458 DEBUG (ScheduledTrigger-122-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10005_solr
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 178528 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178564 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 178565 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 178565 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 178565 DEBUG (ScheduledTrigger-122-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4]
[...truncated too long message...]
987-thread-3) [n:127.0.0.1:42668_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.hdfsbackuprestore.shard2.leader, tag=3f9292a8
[junit4] 2> 6718968 INFO (coreCloseExecutor-13986-thread-1) [n:127.0.0.1:40759_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_t8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.hdfsbackuprestore.shard2.leader, tag=a7c2bfd
[junit4] 2> 6718999 INFO (jetty-closer-57564-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72438645494841350-127.0.0.1:40759_solr-n_0000000000) closing
[junit4] 2> 6718999 INFO (OverseerStateUpdate-72438645494841350-127.0.0.1:40759_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40759_solr
[junit4] 2> 6719000 WARN (OverseerAutoScalingTriggerThread-72438645494841350-127.0.0.1:40759_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 6719003 INFO (jetty-closer-57564-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@381f9003{/solr,null,UNAVAILABLE}
[junit4] 2> 6719003 INFO (jetty-closer-57564-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 6719004 INFO (jetty-closer-57564-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6298e1cc{/solr,null,UNAVAILABLE}
[junit4] 2> 6719005 INFO (jetty-closer-57564-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 6719006 ERROR (SUITE-TestHdfsCloudBackupRestore-seed#[93925E63CC51A4B4]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 6719014 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[93925E63CC51A4B4]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45419 45419
[junit4] 2> 6719074 INFO (Thread-10622) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45419 45419
[junit4] 2> 6719075 WARN (Thread-10622) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections/hdfsbackuprestore/terms/shard2
[junit4] 2> 5 /solr/collections/hdfsbackuprestore/terms/shard1
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 3 /solr/collections/hdfsbackuprestore_restored/terms/shard1
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/configs/conf1
[junit4] 2> 2 /solr/collections/hdfsbackuprestore_restored/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 16 /solr/collections/hdfsbackuprestore/state.json
[junit4] 2> 15 /solr/collections/hdfsbackuprestore_restored/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_93925E63CC51A4B4-001
[junit4] 2> Apr 11, 2018 8:03:15 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 39 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {shard_s=PostingsFormat(name=LuceneFixedGap), id=FST50}, docValues:{}, maxPointsInLeafNode=1843, maxMBSortInHeap=7.6243392989430845, sim=RandomSimilarity(queryNorm=false): {}, locale=uk-UA, timezone=Africa/Asmera
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=149539104,total=479723520
[junit4] 2> NOTE: All tests run in this JVM: [TestSmileRequest, SchemaVersionSpecificBehaviorTest, TestReload, TestNumericTokenStream, ZkShardTermsTest, RegexBytesRefFilterTest, TestDistributedGrouping, ResourceLoaderTest, TestRTGBase, TestScoreJoinQPScore, TestAuthenticationFramework, TestSweetSpotSimilarityFactory, ReplicaListTransformerTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestFuzzyAnalyzedSuggestions, FastVectorHighlighterTest, TestOnReconnectListenerSupport, XsltUpdateRequestHandlerTest, SyncSliceTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestReversedWildcardFilterFactory, TestUseDocValuesAsStored, TestSkipOverseerOperations, TestFoldingMultitermQuery, TestLuceneIndexBackCompat, DistributedQueueTest, TestMaxScoreQueryParser, TestCustomStream, TestBackupRepositoryFactory, TestJsonFacetsWithNestedObjects, SuggestComponentContextFilterQueryTest, TestPartialUpdateDeduplication, PeerSyncTest, TestIntervalFaceting, TestZkAclsWithHadoopAuth, TestCursorMarkWithoutUniqueKey, CurrencyFieldTypeTest, BasicAuthStandaloneTest, SuggesterTest, OverseerModifyCollectionTest, TestNumericTerms64, TestCloudPseudoReturnFields, WrapperMergePolicyFactoryTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestReloadDeadlock, TestDelegationWithHadoopAuth, TestSchemaNameResource, TestSubQueryTransformerDistrib, HdfsUnloadDistributedZkTest, JsonLoaderTest, SignatureUpdateProcessorFactoryTest, TestSimpleTextCodec, TestRTimerTree, TestExportWriter, TestFieldCacheSort, TestSchemaManager, CollectionTooManyReplicasTest, TestComplexPhraseLeadingWildcard, SOLR749Test, ZkControllerTest, LeaderFailoverAfterPartitionTest, TestSolrJ, SortSpecParsingTest, TestDistribDocBasedVersion, NodeAddedTriggerTest, BufferStoreTest, TestPayloadScoreQParserPlugin, LargeFieldTest, TestReplicationHandler, TestDefaultStatsCache, TestPKIAuthenticationPlugin, TestCloudNestedDocsSort, DirectSolrConnectionTest, CloudExitableDirectoryReaderTest, MetricsConfigTest, AnalyticsQueryTest, TestBM25SimilarityFactory, TestSearchPerf, TestExpandComponent, EnumFieldTest, TriggerCooldownIntegrationTest, TestSimDistributedQueue, TestMultiValuedNumericRangeQuery, TestNRTOpen, SecurityConfHandlerTest, SuggesterFSTTest, HdfsChaosMonkeyNothingIsSafeTest, TestHighFrequencyDictionaryFactory, ExitableDirectoryReaderTest, MetricTriggerIntegrationTest, ParsingFieldUpdateProcessorsTest, TestBlendedInfixSuggestions, TestRecovery, LegacyCloudClusterPropTest, TestLeaderInitiatedRecoveryThread, TestCoreAdminApis, IndexSchemaTest, HighlighterTest, SchemaApiFailureTest, TestRestManager, TestQueryUtils, TestFieldCacheWithThreads, TestConfigSetsAPIZkFailure, SolrCLIZkUtilsTest, TestSortByMinMaxFunction, TestTolerantUpdateProcessorRandomCloud, TestInPlaceUpdatesStandalone, DocumentBuilderTest, DistributedFacetPivotWhiteBoxTest, HdfsRecoverLeaseTest, DistributedFacetSimpleRefinementLongTailTest, TestSolrConfigHandlerConcurrent, SearchRateTriggerTest, TestClusterProperties, MultiSolrCloudTestCaseTest, TestRawTransformer, ShardSplitTest, ActionThrottleTest, TermsComponentTest, TestDocTermOrds, ConnectionManagerTest, DistributedIntervalFacetingTest, TestSolrFieldCacheBean, CoreAdminOperationTest, CloneFieldUpdateProcessorFactoryTest, V2StandaloneTest, TestExecutePlanAction, QueryElevationComponentTest, SortByFunctionTest, TestSolrCloudSnapshots, SliceStateTest, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, TestSort, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, TestBadConfig, SoftAutoCommitTest, CSVRequestHandlerTest, TestLFUCache, TestCrossCoreJoin, TestDistributedMissingSort, TokenizerChainTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerSchemaAPI, ConnectionReuseTest, AssignBackwardCompatibilityTest, CleanupOldIndexTest, ConfigSetsAPITest, CreateCollectionCleanupTest, CreateRoutedAliasTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DocValuesNotIndexedTest, ForceLeaderTest, LIRRollingUpdatesTest, OverseerTaskQueueTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestPullReplicaErrorHandling, TestRequestForwarding, TestSSLRandomization, TestSegmentSorting, TestShortCircuitedRequests, TestSolrCloudWithSecureImpersonation, TestStressLiveNodes, TriLevelCompositeIdRoutingTest, CollectionReloadTest, ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, SimpleCollectionCreateDeleteTest, TestCollectionAPI, TestCollectionsAPIViaSolrCloudCluster, TestHdfsCloudBackupRestore]
[junit4] Completed [604/800 (3!)] on J1 in 25.97s, 1 test, 1 error <<< FAILURES!
[...truncated 657 lines...]
[junit4] JVM J1: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20180411_181116_5877357961297125018620.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid6943.hprof ...
[junit4] Heap dump file created [475496812 bytes in 3.465 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 9284 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:651: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid6943.hprof
Total time: 495 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any