You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/06/19 21:40:29 UTC

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 80 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/80/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([A35E453ADC4267E8:FE165BB31384C1A7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13409 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A35E453ADC4267E8-001/init-core-data-001
   [junit4]   2> 1482207 DEBUG (SUITE-TestTriggerIntegration-seed#[A35E453ADC4267E8]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10016_solr
   [junit4]   2> 1482207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1482208 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1482208 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1482209 INFO  (SUITE-TestTriggerIntegration-seed#[A35E453ADC4267E8]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1482216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1482223 DEBUG (ScheduledTrigger-3680-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482231 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1482231 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1482231 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1482233 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482233 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482238 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482238 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1482238 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482258 DEBUG (ScheduledTrigger-3685-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482279 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482342 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482347 DEBUG (simCloudManagerPool-3684-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482347 DEBUG (simCloudManagerPool-3684-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1482350 DEBUG (simCloudManagerPool-3684-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482350 DEBUG (simCloudManagerPool-3684-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1482351 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1482351 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482351 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482352 DEBUG (simCloudManagerPool-3684-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482353 DEBUG (simCloudManagerPool-3684-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1482353 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1482353 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1482363 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10018_solr at time 29703443679716759
   [junit4]   2> 1482383 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10018_solr] added at times [29703443679716759], now=29703444693308009
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"69871fe922a597Te116fkt8tfq9pffeimnm8zqzq",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703443679716759,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703443679716759],
   [junit4]   2>     "nodeNames":["127.0.0.1:10018_solr"]}}
   [junit4]   2> 1482396 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1482419 DEBUG (simCloudManagerPool-3684-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482419 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10018_solr
   [junit4]   2> 1482419 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"69871fe922a597Te116fkt8tfq9pffeimnm8zqzq",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703443679716759,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703443679716759],
   [junit4]   2>     "_enqueue_time_":29703444937672159,
   [junit4]   2>     "nodeNames":["127.0.0.1:10018_solr"]}}
   [junit4]   2> 1482421 DEBUG (simCloudManagerPool-3684-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 109 ms for event id=69871fe922a597Te116fkt8tfq9pffeimnm8zqzq
   [junit4]   2> 1482521 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10019_solr at time 29703451196576559
   [junit4]   2> 1482541 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10019_solr] added at times [29703451196576559], now=29703452207848959
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703451196576559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703451196576559],
   [junit4]   2>     "nodeNames":["127.0.0.1:10019_solr"]}}
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1482543 DEBUG (simCloudManagerPool-3684-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482543 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10019_solr
   [junit4]   2> 1482543 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703451196576559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703451196576559],
   [junit4]   2>     "_enqueue_time_":29703452219375509,
   [junit4]   2>     "nodeNames":["127.0.0.1:10019_solr"]}}
   [junit4]   2> 1482566 DEBUG (simCloudManagerPool-3684-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1160 ms for event id=698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt
   [junit4]   2> 1482585 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		4
   [junit4]   2> ## Empty nodes:	3
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             4
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_cooldown_trigger
   [junit4]   2> ##		- STARTED         2
   [junit4]   2> ##		- SUCCEEDED       2
   [junit4]   2> 
   [junit4]   2> 1482589 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10017_solr
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10019_solr
   [junit4]   2> 1482590 INFO  (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1482590 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1482613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr]
   [junit4]   2> 1482618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1482620 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482641 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482661 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482682 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482702 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482722 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482723 DEBUG (simCloudManagerPool-3687-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482723 DEBUG (simCloudManagerPool-3687-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 1482742 DEBUG (simCloudManagerPool-3687-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482742 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482742 DEBUG (simCloudManagerPool-3687-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 1482743 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr]
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction}], enabled=true}
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 1482743 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482743 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482750 DEBUG (simCloudManagerPool-3687-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482750 DEBUG (simCloudManagerPool-3687-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 1482751 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 1482751 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 1482758 DEBUG (simCloudManagerPool-3687-thread-4) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482758 DEBUG (simCloudManagerPool-3687-thread-4) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 1482762 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482763 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 1482763 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 1482764 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482764 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482783 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10020_solr at time 29703464298133159
   [junit4]   2> 1482803 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482804 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482804 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10020_solr] added at times [29703464298133159], now=29703467958408109
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703464298133159,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703464298133159],
   [junit4]   2>     "nodeNames":["127.0.0.1:10020_solr"]}}
   [junit4]   2> 1482859 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1482947 DEBUG (simCloudManagerPool-3687-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482983 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10020_solr
   [junit4]   2> 1482984 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703464298133159,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703464298133159],
   [junit4]   2>     "_enqueue_time_":29703468074017759,
   [junit4]   2>     "nodeNames":["127.0.0.1:10020_solr"]}}
   [junit4]   2> 1483022 DEBUG (simCloudManagerPool-3687-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1964 ms for event id=698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10021_solr at time 29703481285195309
   [junit4]   2> 1483143 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483144 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483144 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483163 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483164 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483164 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10021_solr] added at times [29703481285195309], now=29703484318094859
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1483222 DEBUG (simCloudManagerPool-3687-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483222 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10021_solr
   [junit4]   2> 1483222 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "_enqueue_time_":29703484330833309,
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> 1483244 DEBUG (simCloudManagerPool-3687-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483245 WARN  (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> java.lang.Exception: Error executing action: test1 for trigger event: {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "_enqueue_time_":29703484330833309,
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.RuntimeException: failure
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction.process(TestTriggerIntegration.java:911) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1151 ms for event id=698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz
   [junit4]   2> 1483264 INFO  (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		4
   [junit4]   2> ## Empty nodes:	3
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        4
   [junit4]   2> ##		- update             4
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger
   [junit4]   2> ##		- FAILED          1
   [junit4]   2> ##		- STARTED         2
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> 
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10020_solr
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10018_solr
   [junit4]   2> 1483269 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1483269 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1483281 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1483283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 1483284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion -1
   [junit4]   2> 1483284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 1483291 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr]
   [junit4]   2> 1483302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 1483303 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483323 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483457 DEBUG (simCloudManagerPool-3690-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1483457 DEBUG (simCloudManagerPool-3690-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 1483475 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483475 WARN  (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
   [junit4]   2> 1483476 ERROR (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
   [junit4]   2> java.lang.RuntimeException: Trigger has been closed
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:588) [java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1483491 DEBUG (simCloudManagerPool-3690-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1483491 DEBUG (simCloudManagerPool-3690-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 1483491 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 1483491 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1483491 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr]
   [junit4]   2> 1483492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1483492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10022_solr at time 29703499699450559
   [junit4]   2> 1483496 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483512 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483512 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483516 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483533 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483533 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483536 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483553 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483553 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483556 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483573 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483573 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483576 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483594 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483594 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483597 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483614 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483614 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483617 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483634 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483634 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483637 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483655 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483655 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483657 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483675 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483675 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483677 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10022_solr] added at times [29703499699450559], now=29703509880923259
   [junit4]   2> 1483696 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703499699450559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703499699450559],
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 1483696 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1483739 DEBUG (simCloudManagerPool-3690-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483748 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10022_solr
   [junit4]   2> 1483748 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703499699450559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703499699450559],
   [junit4]   2>     "_enqueue_time_":29703509929516359,
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 1483763 DEBUG (simCloudManagerPool-3690-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1315 ms for event id=69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 29703518847671609
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483967 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1483967 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1483968 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1483968 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1483968 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion -1
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1483984 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10023_solr
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 1483985 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483990 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 1483990 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483990 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484005 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484010 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484010 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484025 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484031 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484031 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484045 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484051 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484051 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484066 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484071 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484071 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484086 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10023_solr] added at times [29703518847671609], now=29703529674733009
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698731697e9d39Te116fkt8tfq9pffeimnm8zr05",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703518847671609,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703518847671609],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 1484092 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1484106 DEBUG (simCloudManagerPool-3693-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1484115 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 1484115 DEBUG (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698731697e9d39Te116fkt8tfq9pffeimnm8zr05",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703518847671609,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703518847671609],
   [junit4]   2>     "_enqueue_time_":29703529700632759,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 1484116 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		4
   [junit4]   2> ## Empty nodes:	3
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        2
   [junit4]   2> ##		- update             3
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger
   [junit4]   2> ##		- STARTED         2
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> 
   [junit4]   2> 1484168 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10022_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10016_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10021_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10022_solr
   [junit4]   2> 1484169 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1484169 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1484170 WARN  (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:756) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:321) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	... 10 more
   [junit4]   2> 1484171 DEBUG (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 2777 ms for event id=698731697e9d39Te116fkt8tfq9pffeimnm8zr05
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion -1
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
   [junit4]   2> 1484182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1484184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1484184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
   [junit4]   2> 1484185 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484205 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484225 DEBUG (ScheduledTrigger-3697-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484275 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484295 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484298 DEBUG (simCloudManagerPool-3696-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1484298 DEBUG (simCloudManagerPool-3696-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
   [junit4]   2> 1484311 INFO  (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
   [junit4]   2> 1484311 INFO  (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
   [junit4]   2> 1484312 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484312 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1484316 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10024_solr at time 29703541729499959
   [junit4]   2> 1484336 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10024_solr] added at times [29703541729499959], now=29703542747135659
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
   [junit4]   2> 1484400 DEBUG (simCloudManagerPool-3696-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node4":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10024_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1484414 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10024_solr
   [junit4]   2> 1484414 DEBUG (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "_enqueue_time_":29703542773519459,
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484414 INFO  (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "_enqueue_time_":29703542773519459,
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484414 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1484414 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1484416 WARN  (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1484416 DEBUG (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 101 ms for event id=698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1484417 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.js

[...truncated too long message...]

le = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 196ms :: artifacts dl 13ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 174 minutes 16 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 98 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/98/

10 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive

Error Message:
Jetty Connector is not open: -2

Stack Trace:
java.lang.IllegalStateException: Jetty Connector is not open: -2
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:CBE7ABA55E47DF5A]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getBaseUrl(JettySolrRunner.java:499)
	at org.apache.solr.cloud.MiniSolrCloudCluster.getReplicaJetty(MiniSolrCloudCluster.java:539)
	at org.apache.solr.cloud.LeaderTragicEventTest.corruptLeader(LeaderTragicEventTest.java:100)
	at org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive(LeaderTragicEventTest.java:150)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Error from server at https://127.0.0.1:57325/solr: collection already exists: collection1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:57325/solr: collection already exists: collection1
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:C607B808CC440B3A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds

Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10024_solr, 127.0.0.1:10023_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testMixedBounds_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10024_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "core":"testMixedBounds_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10023_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1531913342490000200",       "replicas":{         "core_node1":{           "core":"testMixedBounds_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10024_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495},         "core_node2":{           "core":"testMixedBounds_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10023_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1531913342518424850",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10023_solr",           "base_url":"http://127.0.0.1:10023/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247},         "core_node9":{           "core":"testMixedBounds_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10024_solr",           "base_url":"http://127.0.0.1:10024/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1531913342518237450",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10024_solr",           "base_url":"http://127.0.0.1:10024/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248},         "core_node8":{           "core":"testMixedBounds_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10023_solr",           "base_url":"http://127.0.0.1:10023/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248}}}}}

Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10024_solr, 127.0.0.1:10023_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10024_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10023_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1531913342490000200",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10024_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10023_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1531913342518424850",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10023_solr",
          "base_url":"http://127.0.0.1:10023/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10024_solr",
          "base_url":"http://127.0.0.1:10024/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1531913342518237450",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10024_solr",
          "base_url":"http://127.0.0.1:10024/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10023_solr",
          "base_url":"http://127.0.0.1:10023/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:44D0387F2F036D98]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:585)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/25)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10024_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10023_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1531913342490000200",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10024_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10023_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1531913342518424850",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10023_solr",
          "base_url":"http://127.0.0.1:10023/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10024_solr",
          "base_url":"http://127.0.0.1:10024/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1531913342518237450",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10024_solr",
          "base_url":"http://127.0.0.1:10024/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10023_solr",
          "base_url":"http://127.0.0.1:10023/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
	... 40 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:2D98B150FB7715EF]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost

Error Message:
did not finish processing all events in time: started=5, finished=4

Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:F146492CE1520344]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:525)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:378)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:4E6533124BB5C2A8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState(TestTriggerIntegration.java:696)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testExistOldBufferLog

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:10039A87EC77F64B]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.junit.Assert.assertFalse(Assert.java:79)
	at org.apache.solr.search.TestRecovery.testExistOldBufferLog(TestRecovery.java:1073)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.test

Error Message:
Wrong doc count on shard1_0. See SOLR-5309 expected:<208> but was:<207>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0. See SOLR-5309 expected:<208> but was:<207>
	at __randomizedtesting.SeedInfo.seed([4E5387D262B866C2:C607B808CC440B3A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:968)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.splitByUniqueKeyTest(ShardSplitTest.java:751)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.test(ShardSplitTest.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=32165349941167527, stage=STARTED, actionName='null', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351154622827, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351200517577, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351225715477, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165360041033077, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/7246362e484949T4ekf6hrvz9evkidxee1sijw5h/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/7246362e484949T4ekf6hrvz9evkidxee1sijw5h/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165360070407377, stage=SUCCEEDED, actionName='null', event={   "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",   "source":"index_size_trigger2",   "eventTime":32165345863682377,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":32165349935679327,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369324343377, stage=STARTED, actionName='null', event={   "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",   "source":"index_size_trigger2",   "eventTime":32165365214472177,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":32165369316368427,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369476323327, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",   "source":"index_size_trigger2",   "eventTime":32165365214472177,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":32165369316368427,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369540945777, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",   "source":"index_size_trigger2",   "eventTime":32165365214472177,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":32165369316368427,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369566210677, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",   "source":"index_size_trigger2",   "eventTime":32165365214472177,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":32165369316368427,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=32165349941167527, stage=STARTED, actionName='null', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351154622827, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351200517577, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165351225715477, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165360041033077, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/7246362e484949T4ekf6hrvz9evkidxee1sijw5h/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/7246362e484949T4ekf6hrvz9evkidxee1sijw5h/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165360070407377, stage=SUCCEEDED, actionName='null', event={
  "id":"7246362e484949T4ekf6hrvz9evkidxee1sijw5h",
  "source":"index_size_trigger2",
  "eventTime":32165345863682377,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":32165349935679327,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369324343377, stage=STARTED, actionName='null', event={
  "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",
  "source":"index_size_trigger2",
  "eventTime":32165365214472177,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":32165369316368427,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369476323327, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",
  "source":"index_size_trigger2",
  "eventTime":32165365214472177,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"base_url\":\"http://127.0.0.1:10024/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10024_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":32165369316368427,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=32165369540945777, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"72463aafade7f1T4ekf6hrvz9evkidxee1sijw5k",
  "source":"index_size_trigger2",
  "eventTime":32165365214472177,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10023_solr\",\n    \"base_url\":\"http://127.0.0.1:10023/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{

[...truncated too long message...]

kins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null126040084
     [copy] Copying 241 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null126040084
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null126040084

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 95ms :: artifacts dl 7ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 160 minutes 24 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 97 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/97/

7 tests failed.
FAILED:  org.apache.lucene.document.TestLatLonShape.testBasicIntersects

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
	at __randomizedtesting.SeedInfo.seed([B9E6444F46A67750:8E9BB7BA7A31D42E]:0)
	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
	at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:215)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
	at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
	at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.writeField(AssertingPointsFormat.java:137)
	at org.apache.lucene.index.PointValuesWriter.flush(PointValuesWriter.java:183)
	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:215)
	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:141)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:470)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:554)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:719)
	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:492)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
	at org.apache.lucene.document.TestLatLonShape.testBasicIntersects(TestLatLonShape.java:111)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes

Error Message:
unexpected shard state expected:<inactive> but was:<active>

Stack Trace:
java.lang.AssertionError: unexpected shard state expected:<inactive> but was:<active>
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:73EAA0308933A352]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.verifyShard(ShardSplitTest.java:380)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes(ShardSplitTest.java:372)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.test

Error Message:
Wrong doc count on shard1_0. See SOLR-5309 expected:<208> but was:<215>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_0. See SOLR-5309 expected:<208> but was:<215>
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:437DCB4ADB141BDF]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:968)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.splitByUniqueKeyTest(ShardSplitTest.java:751)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.test(ShardSplitTest.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testExecute

Error Message:
last state: DocCollection(testExecute//clusterstate.json/23)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{"shard1":{       "replicas":{         "core_node1":{           "core":"testExecute_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node2":{           "core":"testExecute_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10002_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "node_name":"127.0.0.1:10001_solr",           "core":"testExecute_shard1_replica_n3",           "state":"active",           "INDEX.sizeInBytes":10240,           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6}},       "range":"80000000-7fffffff",       "state":"active"}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testExecute//clusterstate.json/23)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{"shard1":{
      "replicas":{
        "core_node1":{
          "core":"testExecute_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node2":{
          "core":"testExecute_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10002_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "node_name":"127.0.0.1:10001_solr",
          "core":"testExecute_shard1_replica_n3",
          "state":"active",
          "INDEX.sizeInBytes":10240,
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6}},
      "range":"80000000-7fffffff",
      "state":"active"}}}
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:FA99E69D671F89A0]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testExecute(TestExecutePlanAction.java:154)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "overlay":{     "znodeVersion":0,     "runtimeLib":{"colltest":{         "name":"colltest",         "version":1}}}},  from server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "overlay":{
    "znodeVersion":0,
    "runtimeLib":{"colltest":{
        "name":"colltest",
        "version":1}}}},  from server:  null
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:1364D9C78235D387]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:595)
	at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:79)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/131)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testSplitIntegration_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10003_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "core":"testSplitIntegration_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10002_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1531824327877547800",       "replicas":{         "core_node1":{           "core":"testSplitIntegration_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":25,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":22740,           "node_name":"127.0.0.1:10003_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.117827534675598E-5,           "SEARCHER.searcher.numDocs":25},         "core_node2":{           "core":"testSplitIntegration_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":25,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":22740,           "node_name":"127.0.0.1:10002_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.117827534675598E-5,           "SEARCHER.searcher.numDocs":25}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1531824327878076650",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":12,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":16240,           "node_name":"127.0.0.1:10002_solr",           "base_url":"http://127.0.0.1:10002/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.5124678611755371E-5,           "SEARCHER.searcher.numDocs":12},         "core_node9":{           "core":"testSplitIntegration_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":12,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":16240,           "node_name":"127.0.0.1:10003_solr",           "base_url":"http://127.0.0.1:10003/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.5124678611755371E-5,           "SEARCHER.searcher.numDocs":12}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1531824327877977550",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":13,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":26980,           "node_name":"127.0.0.1:10003_solr",           "base_url":"http://127.0.0.1:10003/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.512708306312561E-5,           "SEARCHER.searcher.numDocs":13},         "core_node8":{           "core":"testSplitIntegration_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":13,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":26980,           "node_name":"127.0.0.1:10002_solr",           "base_url":"http://127.0.0.1:10002/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.512708306312561E-5,           "SEARCHER.searcher.numDocs":13}}}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/131)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testSplitIntegration_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10003_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testSplitIntegration_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10002_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1531824327877547800",
      "replicas":{
        "core_node1":{
          "core":"testSplitIntegration_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":25,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":22740,
          "node_name":"127.0.0.1:10003_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.117827534675598E-5,
          "SEARCHER.searcher.numDocs":25},
        "core_node2":{
          "core":"testSplitIntegration_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":25,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":22740,
          "node_name":"127.0.0.1:10002_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.117827534675598E-5,
          "SEARCHER.searcher.numDocs":25}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1531824327878076650",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":12,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":16240,
          "node_name":"127.0.0.1:10002_solr",
          "base_url":"http://127.0.0.1:10002/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.5124678611755371E-5,
          "SEARCHER.searcher.numDocs":12},
        "core_node9":{
          "core":"testSplitIntegration_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":12,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":16240,
          "node_name":"127.0.0.1:10003_solr",
          "base_url":"http://127.0.0.1:10003/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.5124678611755371E-5,
          "SEARCHER.searcher.numDocs":12}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1531824327877977550",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":13,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":26980,
          "node_name":"127.0.0.1:10003_solr",
          "base_url":"http://127.0.0.1:10003/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.512708306312561E-5,
          "SEARCHER.searcher.numDocs":13},
        "core_node8":{
          "core":"testSplitIntegration_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":13,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":26980,
          "node_name":"127.0.0.1:10002_solr",
          "base_url":"http://127.0.0.1:10002/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.512708306312561E-5,
          "SEARCHER.searcher.numDocs":13}}}}}
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:F2A74DD05A17BFD9]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:313)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([CB29F49075E87627:400E274134EEDDA3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 9555 lines...]
   [junit4] Suite: org.apache.lucene.document.TestLatLonShape
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLatLonShape -Dtests.method=testBasicIntersects -Dtests.seed=B9E6444F46A67750 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ar-EG -Dtests.timezone=NZ-CHAT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   57.1s J0 | TestLatLonShape.testBasicIntersects <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B9E6444F46A67750:8E9BB7BA7A31D42E]:0)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    > 	at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > 	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > 	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    > 	at org.apache.lucene.util.bkd.OfflinePointReader.split(OfflinePointReader.java:215)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1787)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1801)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1814)
   [junit4]    > 	at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1004)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1$1.writeField(RandomCodec.java:139)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.writeField(AssertingPointsFormat.java:137)
   [junit4]    > 	at org.apache.lucene.index.PointValuesWriter.flush(PointValuesWriter.java:183)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:215)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:141)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:470)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:554)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:719)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:492)
   [junit4]    > 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:103)
   [junit4]    > 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:79)
   [junit4]    > 	at org.apache.lucene.document.TestLatLonShape.testBasicIntersects(TestLatLonShape.java:111)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=173, maxMBSortInHeap=5.935764453923064, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2e2bcde4), locale=ar-EG, timezone=NZ-CHAT
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=329883400,total=344457216
   [junit4]   2> NOTE: All tests run in this JVM: [TestPayloadSpanUtil, TestIDVersionPostingsFormat, TestLatLonBoundingBoxQueries, TestDocValuesNumbersQuery, TestFloatPointNearestNeighbor, TestHalfFloatPoint, TestLatLonDocValuesField, TestLatLonPoint, TestLatLonPointDistanceSort, TestLatLonShape]
   [junit4] Completed [18/24 (1!)] on J0 in 57.17s, 2 tests, 1 error, 1 skipped <<< FAILURES!

[...truncated 16 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/build/sandbox/test/temp/junit4-J0-20180715_194722_5937048729078580913512.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/heapdumps/java_pid16781.hprof ...
   [junit4] Heap dump file created [434865251 bytes in 2.027 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 2640 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction
   [junit4]   2> 74147 INFO  (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestExecutePlanAction_CB29F49075E87627-001/init-core-data-001
   [junit4]   2> 74148 WARN  (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 74149 INFO  (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 74176 INFO  (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 74437 DEBUG (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 74439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 74440 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 74441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 74441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 74450 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 74470 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 74470 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 74471 DEBUG (ScheduledTrigger-146-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 74485 INFO  (SUITE-TestExecutePlanAction-seed#[CB29F49075E87627]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 74490 INFO  (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testIntegration
   [junit4]   2> 74490 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 74491 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 74492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 74492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 74492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 74493 INFO  (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 74493 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 74535 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 74545 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 74545 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 74556 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 74556 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 74599 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 74605 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 74605 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 74608 DEBUG (ScheduledTrigger-151-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74650 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74679 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74710 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74731 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74751 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74772 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74792 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74812 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74833 DEBUG (ScheduledTrigger-151-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74840 DEBUG (simCloudManagerPool-150-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 74840 DEBUG (simCloudManagerPool-150-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 74848 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 74849 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 74849 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 74849 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 74849 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 74938 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74960 DEBUG (ScheduledTrigger-151-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 74961 DEBUG (ScheduledTrigger-151-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 74964 DEBUG (simCloudManagerPool-150-thread-2) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testIntegration
   [junit4]   2> 74981 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 74981 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75001 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 75001 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75013 DEBUG (simCloudManagerPool-150-thread-2) [    ] o.a.s.c.a.c.Assign Identify nodes using default
   [junit4]   2> 75021 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 75022 DEBUG (ScheduledTrigger-151-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75042 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 75042 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75062 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 75062 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75086 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 75086 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 75112 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10001_solr
   [junit4]   2> 75118 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75118 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 75119 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
   [junit4]   2> 75119 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 75119 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
   [junit4]   2> 75121 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75122 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75122 DEBUG (simCloudManagerPool-150-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testIntegration / shard1: {"core_node1":{
   [junit4]   2>     "core":"testIntegration_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testIntegration"}}
   [junit4]   2> 75123 DEBUG (simCloudManagerPool-150-thread-6) [    ] o.a.s.c.ActionThrottle The last leader attempt started 142ms ago.
   [junit4]   2> 75123 INFO  (simCloudManagerPool-150-thread-6) [    ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 4857ms
   [junit4]   2> 75123 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75124 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75125 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75126 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75128 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75129 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75130 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75131 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75132 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75133 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75134 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75135 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75137 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75138 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75139 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 75139 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10000_solr]
   [junit4]   2> 75141 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7176316d4310f3T1sfaeiikkb124ovarlph3b8q2",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":31936627026890995,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31936627026890995],
   [junit4]   2>     "nodeNames":["127.0.0.1:10000_solr"]}}
   [junit4]   2> 75141 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75141 DEBUG (ScheduledTrigger-151-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 75142 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75144 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75145 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75146 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75147 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75148 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75149 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75150 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75151 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75153 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75154 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75155 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75156 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75157 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75157 DEBUG (simCloudManagerPool-150-thread-7) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 75158 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75159 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75161 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75162 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75163 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75164 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75165 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75166 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75167 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75168 DEBUG (simCloudManagerPool-150-thread-7) [    ] o.a.s.c.a.c.Assign Identify nodes using default
   [junit4]   2> 75169 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75171 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75172 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75173 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75174 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75177 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75178 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75179 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75180 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75181 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75182 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75193 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75194 DEBUG (simCloudManagerPool-150-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 75207 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75209 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75214 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75215 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75216 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75218 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75219 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75220 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75221 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75222 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75226 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75227 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75228 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75230 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75231 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75232 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75233 DEBUG (AutoscalingActionExecutor-152-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7176316d4310f3T1sfaeiikkb124ovarlph3b8q2",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":31936627026890995,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31936627026890995],
   [junit4]   2>     "_enqueue_time_":31936628171039495,
   [junit4]   2>     "nodeNames":["127.0.0.1:10000_solr"]}}
   [junit4]   2> 75233 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75234 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75235 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75236 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75237 DEBUG (AutoscalingActionExecutor-152-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"7176316d4310f3T1sfaeiikkb124ovarlph3b8q2",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":31936627026890995,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31936627026890995],
   [junit4]   2>     "_enqueue_time_":31936628171039495,
   [junit4]   2>     "nodeNames":["127.0.0.1:10000_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 75238 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75239 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75240 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75241 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75242 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75243 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75244 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75246 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75247 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75248 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75249 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75250 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75251 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75252 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75254 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75255 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75256 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75257 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75258 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75259 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75262 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75263 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75264 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75265 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75267 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75268 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75269 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75270 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75271 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75272 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75273 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75275 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75276 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75277 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75278 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75279 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75280 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75281 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75283 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75284 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75285 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75286 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75287 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75288 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75289 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75291 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75297 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75298 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75300 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75301 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75302 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75303 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75304 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75306 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75307 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75308 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E87627]) [    ] o.a.s.c.CloudTestUtils -- wrong number of active replicas in slice shard1, expected=2, found=1
   [junit4]   2> 75310 DEBUG (TEST-TestExecutePlanAction.testIntegration-seed#[CB29F49075E876

[...truncated too long message...]

27.0.0.1:55433__gp c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 3277560 INFO  (zkCallback-7988-thread-1) [n:127.0.0.1:55433__gp c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.c.SolrCore [collection1_shard1_replica_t47]  CLOSING SolrCore org.apache.solr.core.SolrCore@46b55cf3
   [junit4]   2> 3277560 INFO  (zkCallback-7988-thread-1) [n:127.0.0.1:55433__gp c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_t47, tag=46b55cf3
   [junit4]   2> 3277560 INFO  (zkCallback-7988-thread-1) [n:127.0.0.1:55433__gp c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@327e893d: rootName = null, domain = solr.core.collection1.shard1.replica_t47, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_t47 / com.codahale.metrics.MetricRegistry@3ed1f31b
   [junit4]   2> 3277569 INFO  (zkCallback-7988-thread-1) [n:127.0.0.1:55433__gp c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=46b55cf3
   [junit4]   2> 3277577 INFO  (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@2d881f0c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 3277577 INFO  (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@d47b255{/_gp,null,UNAVAILABLE}
   [junit4]   2> 3277577 INFO  (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3277579 INFO  (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 53543
   [junit4]   2> 3277579 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3277581 INFO  (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[CB29F49075E87627]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47868 47868
   [junit4]   2> 3277666 INFO  (Thread-8900) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47868 47868
   [junit4]   2> 3277667 WARN  (Thread-8900) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	14	/solr/collections/collection1/terms/shard1_1
   [junit4]   2> 	12	/solr/aliases.json
   [junit4]   2> 	10	/solr/collections/collection1/terms/shard1_0
   [junit4]   2> 	9	/solr/collections/collection1/terms/shard1
   [junit4]   2> 	9	/solr/collections/collection1/terms/shard2
   [junit4]   2> 	8	/solr/collections/collection1/state.json
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	2	/solr/collections/control_collection/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/clusterprops.json
   [junit4]   2> 	12	/solr/clusterstate.json
   [junit4]   2> 	5	/solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	12	/solr/collections
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 	5	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	5	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	5	/solr/overseer/queue-work
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_CB29F49075E87627-001
   [junit4]   2> Jul 15, 2018 8:51:28 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@10537293), locale=mt-MT, timezone=AGT
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=200550784,total=527958016
   [junit4]   2> NOTE: All tests run in this JVM: [PeerSyncWithLeaderAndIndexFingerprintCachingTest, TestCSVLoader, UpdateParamsTest, TestChildDocTransformer, TestCloudNestedDocsSort, DistributedFacetPivotLargeTest, TestQueryTypes, UnloadDistributedZkTest, TestBadConfig, SpellCheckCollatorTest, TestEmbeddedSolrServerConstructors, TestRecoveryHdfs, ConnectionReuseTest, TestPostingsSolrHighlighter, AliasIntegrationTest, TestConfigSets, DistanceFunctionTest, MetricsHandlerTest, ManagedSchemaRoundRobinCloudTest, ResponseHeaderTest, AutoAddReplicasIntegrationTest, TimeRoutedAliasUpdateProcessorTest, TestCloudConsistency, SolrMetricsIntegrationTest, TestScoreJoinQPScore, MigrateRouteKeyTest, ExitableDirectoryReaderTest, TestXmlQParserPlugin, TestCloudSchemaless, TestRawTransformer, ChaosMonkeyNothingIsSafeTest, ReturnFieldsTest, AnalysisAfterCoreReloadTest, CoreAdminRequestStatusTest, SecurityConfHandlerTest, FacetPivotSmallTest, FullHLLTest, DirectUpdateHandlerTest, TestQuerySenderListener, TestSha256AuthenticationProvider, MoveReplicaHDFSTest, TestCloudPseudoReturnFields, ReplicaListTransformerTest, TestMiniSolrCloudClusterSSL, OverriddenZkACLAndCredentialsProvidersTest, TestSolrXml, TestAuthorizationFramework, HttpPartitionTest, BlobRepositoryCloudTest, TestObjectReleaseTracker, TestRandomCollapseQParserPlugin, SolrCoreCheckLockOnStartupTest, TestCrossCoreJoin, TestShardHandlerFactory, SpatialHeatmapFacetsTest, TestConfigOverlay, TestSubQueryTransformerDistrib, TestGeoJSONResponseWriter, TestSolrCLIRunExample, TestSolrCoreProperties, TestTrieFacet, CollectionReloadTest, ResponseBuilderTest, TestSolrDeletionPolicy1, TestComplexPhraseLeadingWildcard, LoggingHandlerTest, HdfsDirectoryTest, TestMinMaxOnMultiValuedField, TestMultiWordSynonyms, AnalyticsMergeStrategyTest, RuleEngineTest, TestDFISimilarityFactory, CdcrBootstrapTest, SpatialFilterTest, ZkStateReaderTest, TestWriterPerf, HdfsWriteToMultipleCollectionsTest, TestUseDocValuesAsStored, TestReqParamsAPI, LeaderElectionContextKeyTest, XmlUpdateRequestHandlerTest, TestPullReplica, TestSolrCloudWithKerberosAlt, TestManagedSchemaThreadSafety, TestCSVResponseWriter, ConfigureRecoveryStrategyTest, TestSolrCoreParser, TestSmileRequest, TestLegacyNumericUtils, DistributedFacetPivotSmallTest, TestPerFieldSimilarity, BlockJoinFacetRandomTest, TestFieldCacheSort, TestSolrCloudWithHadoopAuthPlugin, DirectSolrSpellCheckerTest, ChaosMonkeySafeLeaderWithPullReplicasTest, PreAnalyzedUpdateProcessorTest, CursorPagingTest, TestFieldCacheWithThreads, DeleteShardTest, TestLeaderElectionZkExpiry, SliceStateTest, StatsReloadRaceTest, TestLazyCores, CursorMarkTest, MoreLikeThisHandlerTest, RequestHandlersTest, TestHalfAndHalfDocValues, BadCopyFieldTest, SimpleMLTQParserTest, TestTolerantUpdateProcessorCloud, SignatureUpdateProcessorFactoryTest, RequiredFieldsTest, RestoreTriggerStateTest, DistribDocExpirationUpdateProcessorTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestTriggerIntegration, PreAnalyzedFieldManagedSchemaCloudTest, PeerSyncTest, ChangedSchemaMergeTest, UUIDUpdateProcessorFallbackTest, DistributedSpellCheckComponentTest, TestPHPSerializedResponseWriter, SampleTest, TestLocalFSCloudBackupRestore, SolrJmxReporterCloudTest, TestFieldResource, TestDynamicLoading, TestComputePlanAction, ReplaceNodeTest, ConcurrentCreateRoutedAliasTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DeleteNodeTest, DistributedQueueTest, DistributedVersionInfoTest, FullSolrCloudDistribCmdsTest, HealthCheckHandlerTest, HttpPartitionOnCommitTest, LeaderFailureAfterFreshStartTest, LeaderVoteWaitTimeoutTest, MultiSolrCloudTestCaseTest, MultiThreadedOCPTest, OverseerTest, RecoveryZkTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, RollingRestartTest, SaslZkACLProviderTest, ShardRoutingCustomTest, ShardRoutingTest, SolrXmlInZkTest, SyncSliceTest, TestClusterProperties, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDistribDocBasedVersion, TestDistributedMap, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestRequestForwarding, TestSSLRandomization, TestSegmentSorting, TestSolrCloudWithDelegationTokens, TestStressInPlaceUpdates, ZkControllerTest, ZkSolrClientTest, ConcurrentDeleteAndCreateCollectionTest, HdfsCollectionsAPIDistributedZkTest, ShardSplitTest]
   [junit4] Completed [542/824 (4!)] on J0 in 432.35s, 7 tests, 3 failures <<< FAILURES!

[...truncated 9650 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/build.xml:642: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid16781.hprof

Total time: 141 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-BadApples-Tests-master #96 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 96 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/96/

7 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive

Error Message:
Jetty Connector is not open: -2

Stack Trace:
java.lang.IllegalStateException: Jetty Connector is not open: -2
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:8C299D0908D6E455]:0)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.getBaseUrl(JettySolrRunner.java:499)
	at org.apache.solr.cloud.MiniSolrCloudCluster.getReplicaJetty(MiniSolrCloudCluster.java:539)
	at org.apache.solr.cloud.LeaderTragicEventTest.corruptLeader(LeaderTragicEventTest.java:100)
	at org.apache.solr.cloud.LeaderTragicEventTest.testOtherReplicasAreNotActive(LeaderTragicEventTest.java:150)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Error from server at http://127.0.0.1:51885/solr: collection already exists: collection1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51885/solr: collection already exists: collection1
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:81C98EA49AD53035]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at http://127.0.0.1:37931/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:37931/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:34451F520CC703BD]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
	at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37931/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=31927328076635458, stage=STARTED, actionName='null', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329238198908, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329291997958, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329311763358, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927337673419258, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927337714014958, stage=SUCCEEDED, actionName='null', event={   "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",   "source":"index_size_trigger2",   "eventTime":31927324900565058,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31927328038400558,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927345964288158, stage=STARTED, actionName='null', event={   "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",   "source":"index_size_trigger2",   "eventTime":31927342847932358,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31927345956290608,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346157030808, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",   "source":"index_size_trigger2",   "eventTime":31927342847932358,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31927345956290608,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346285951158, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",   "source":"index_size_trigger2",   "eventTime":31927342847932358,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31927345956290608,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346407948358, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",   "source":"index_size_trigger2",   "eventTime":31927342847932358,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31927345956290608,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=31927328076635458, stage=STARTED, actionName='null', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329238198908, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329291997958, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927329311763358, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927337673419258, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927337714014958, stage=SUCCEEDED, actionName='null', event={
  "id":"716dbb9b53a042T6edohrm0wgkv3ra6the5zd97t",
  "source":"index_size_trigger2",
  "eventTime":31927324900565058,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31927328038400558,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927345964288158, stage=STARTED, actionName='null', event={
  "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",
  "source":"index_size_trigger2",
  "eventTime":31927342847932358,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31927345956290608,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346157030808, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",
  "source":"index_size_trigger2",
  "eventTime":31927342847932358,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31927345956290608,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346285951158, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",
  "source":"index_size_trigger2",
  "eventTime":31927342847932358,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31927345956290608,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31927346407948358, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"716dbfc912b7c6T6edohrm0wgkv3ra6the5zd97w",
  "source":"index_size_trigger2",
  "eventTime":31927342847932358,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"base_url\":\"http://127.0.0.1:10034/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10034_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31927345956290608,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}] expected:<6> but was:<10>
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:3013083E1BD69433]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:317)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:22626425AE51481D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:324)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:82BA62AF752FF649]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([99DB17E34295DCD:54D5AFF7FBEFFB82]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12604 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> 733755 INFO  (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_99DB17E34295DCD-001/init-core-data-001
   [junit4]   2> 733756 WARN  (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=40 numCloses=40
   [junit4]   2> 733759 INFO  (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 733790 INFO  (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 733814 DEBUG (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10009_solr
   [junit4]   2> 733815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 733815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 733815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 733815 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 733834 INFO  (SUITE-TestTriggerIntegration-seed#[99DB17E34295DCD]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 733843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr]
   [junit4]   2> 733854 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 733854 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 733890 DEBUG (ScheduledTrigger-2081-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 733896 INFO  (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
   [junit4]   2> 733896 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 733897 INFO  (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 733897 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 733898 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 733898 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 733898 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 733898 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 733898 WARN  (ScheduledTrigger-2081-thread-3) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
   [junit4]   2> 733898 ERROR (ScheduledTrigger-2081-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
   [junit4]   2> java.lang.RuntimeException: Trigger has been closed
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:588) [java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 733899 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 733899 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[99DB17E34295DCD]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 733910 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 733910 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 733912 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 733912 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 733935 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr]
   [junit4]   2> 733942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 733942 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 733942 DEBUG (ScheduledTrigger-2086-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 733963 DEBUG (ScheduledTrigger-2086-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 733985 DEBUG (ScheduledTrigger-2086-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 734005 DEBUG (ScheduledTrigger-2086-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 734025 DEBUG (ScheduledTrigger-2086-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 734046 DEBUG (ScheduledTrigger-2086-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 734066 DEBUG (ScheduledTrigger-2086-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Run

[...truncated too long message...]

slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1072008974
     [copy] Copying 241 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1072008974
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1072008974

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 133ms :: artifacts dl 10ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 148 minutes 59 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 95 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/95/

4 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes

Error Message:
unexpected shard state expected:<inactive> but was:<active>

Stack Trace:
java.lang.AssertionError: unexpected shard state expected:<inactive> but was:<active>
	at __randomizedtesting.SeedInfo.seed([94FB101D78C6E874:2C3844BD841D3D01]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.verifyShard(ShardSplitTest.java:380)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes(ShardSplitTest.java:372)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10012_solr should have been deleted

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10012_solr should have been deleted
	at __randomizedtesting.SeedInfo.seed([94FB101D78C6E874:8C41981176F3259B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:866)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([94FB101D78C6E874:1FDCC3CC39C043F0]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([94FB101D78C6E874:C9B30E94B7004E3B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12384 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/init-core-data-001
   [junit4]   2> 82015 WARN  (SUITE-ShardSplitTest-seed#[94FB101D78C6E874]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
   [junit4]   2> 82016 INFO  (SUITE-ShardSplitTest-seed#[94FB101D78C6E874]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 82018 INFO  (SUITE-ShardSplitTest-seed#[94FB101D78C6E874]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 82018 INFO  (SUITE-ShardSplitTest-seed#[94FB101D78C6E874]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 82018 INFO  (SUITE-ShardSplitTest-seed#[94FB101D78C6E874]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 82034 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 82044 INFO  (Thread-61) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 82044 INFO  (Thread-61) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 82244 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkTestServer start zk server on port:39598
   [junit4]   2> 82305 ERROR (Thread-61) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 82431 INFO  (zkConnectionManagerCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 82473 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10334c524580000, likely client has closed socket
   [junit4]   2> 82520 INFO  (zkConnectionManagerCallback-18-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 82546 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 82552 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 82567 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 82571 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 82577 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 82587 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 82603 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 82605 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 82608 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 82611 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 82614 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 82619 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 82795 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 82807 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 82807 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 82807 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 82809 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74e67ad6{/,null,AVAILABLE}
   [junit4]   2> 82819 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1ff0d8ed{HTTP/1.1,[http/1.1]}{127.0.0.1:36329}
   [junit4]   2> 82819 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server Started @82986ms
   [junit4]   2> 82819 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=36329, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/control-001/cores}
   [junit4]   2> 82819 ERROR (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 82819 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 82820 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 82820 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 82820 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 82820 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-12T22:00:36.700Z
   [junit4]   2> 82841 INFO  (zkConnectionManagerCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 82854 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 82854 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/control-001/solr.xml
   [junit4]   2> 82858 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 82859 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 82871 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 83077 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39598/solr
   [junit4]   2> 83115 INFO  (zkConnectionManagerCallback-24-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 83119 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10334c524580003, likely client has closed socket
   [junit4]   2> 83139 INFO  (zkConnectionManagerCallback-26-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 83431 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 83432 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36329_
   [junit4]   2> 83434 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.Overseer Overseer (id=72960040291008516-127.0.0.1:36329_-n_0000000000) starting
   [junit4]   2> 83474 INFO  (zkConnectionManagerCallback-33-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 83477 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 83564 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36329_
   [junit4]   2> 83597 INFO  (zkCallback-32-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 83618 INFO  (zkCallback-25-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 83639 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 83659 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 83685 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 83685 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 83706 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:36329_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/control-001/cores
   [junit4]   2> 83815 INFO  (zkConnectionManagerCallback-39-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 83849 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 83850 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 83884 INFO  (qtp1313541930-203) [n:127.0.0.1:36329_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36329_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 83947 INFO  (OverseerThreadFactory-120-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 84094 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_    x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 84096 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_    x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 85131 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 85146 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 85250 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 85290 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 85291 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 85300 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 85300 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 85306 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4966378496681013]
   [junit4]   2> 85320 WARN  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 85379 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 85379 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 85380 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 85380 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 85382 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6029046529292041]
   [junit4]   2> 85384 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@43ea32d1[control_collection_shard1_replica_n1] main]
   [junit4]   2> 85387 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 85388 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 85388 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 85390 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605823720870379520
   [junit4]   2> 85391 INFO  (searcherExecutor-125-thread-1-processing-n:127.0.0.1:36329_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@43ea32d1[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 85410 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 85423 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 85424 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 85424 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36329/control_collection_shard1_replica_n1/
   [junit4]   2> 85425 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 85426 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36329/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 85426 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 85439 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36329/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 85594 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 85600 INFO  (qtp1313541930-198) [n:127.0.0.1:36329_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1507
   [junit4]   2> 85616 INFO  (qtp1313541930-203) [n:127.0.0.1:36329_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 85697 INFO  (zkCallback-25-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 85951 INFO  (OverseerCollectionConfigSetProcessor-72960040291008516-127.0.0.1:36329_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 86616 INFO  (qtp1313541930-203) [n:127.0.0.1:36329_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36329_&wt=javabin&version=2} status=0 QTime=2734
   [junit4]   2> 86659 INFO  (zkConnectionManagerCallback-44-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 86661 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 86662 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 86665 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 86670 INFO  (qtp1313541930-199) [n:127.0.0.1:36329_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 86691 INFO  (OverseerThreadFactory-120-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 86694 WARN  (OverseerThreadFactory-120-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 86906 INFO  (qtp1313541930-199) [n:127.0.0.1:36329_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 86906 INFO  (qtp1313541930-199) [n:127.0.0.1:36329_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=236
   [junit4]   2> 87033 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001 of type NRT
   [junit4]   2> 87034 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 87035 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 87035 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 87035 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 87036 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c7ffda2{/,null,AVAILABLE}
   [junit4]   2> 87036 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@62d48dcf{HTTP/1.1,[http/1.1]}{127.0.0.1:32845}
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server Started @87203ms
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=32845, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001/cores}
   [junit4]   2> 87037 ERROR (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 87037 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-12T22:00:40.917Z
   [junit4]   2> 87054 INFO  (zkConnectionManagerCallback-46-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 87066 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 87066 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001/solr.xml
   [junit4]   2> 87071 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 87071 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 87072 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 87163 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39598/solr
   [junit4]   2> 87187 INFO  (zkConnectionManagerCallback-50-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 87215 INFO  (zkConnectionManagerCallback-52-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 87227 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 87230 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 87247 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 87247 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32845_
   [junit4]   2> 87255 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 87255 INFO  (zkCallback-32-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 87259 INFO  (zkCallback-25-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 87274 INFO  (zkCallback-51-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 87308 INFO  (zkConnectionManagerCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 87311 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 87313 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 87315 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 87338 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 87349 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 87350 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 87359 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:32845_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001/cores
   [junit4]   2> 87395 INFO  (qtp1104857300-259) [n:127.0.0.1:32845_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:32845_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 87400 INFO  (OverseerCollectionConfigSetProcessor-72960040291008516-127.0.0.1:36329_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 87407 INFO  (OverseerThreadFactory-120-thread-3) [ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:32845_ for creating new replica
   [junit4]   2> 87413 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_    x:collection1_shard2_replica_n41] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 88436 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 88506 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 88650 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 88699 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 88700 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 88700 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 88700 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 88704 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4966378496681013]
   [junit4]   2> 88740 WARN  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 88799 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 88799 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 88800 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 88800 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 88802 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6029046529292041]
   [junit4]   2> 88803 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@264182[collection1_shard2_replica_n41] main]
   [junit4]   2> 88805 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 88806 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 88806 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 88807 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605823724453363712
   [junit4]   2> 88873 INFO  (searcherExecutor-139-thread-1-processing-n:127.0.0.1:32845_ x:collection1_shard2_replica_n41 c:collection1 s:shard2) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@264182[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 88879 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node42=0}, version=0}
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32845/collection1_shard2_replica_n41/
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:32845/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 88887 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 88894 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32845/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 89046 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 89050 INFO  (qtp1104857300-254) [n:127.0.0.1:32845_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1637
   [junit4]   2> 89056 INFO  (qtp1104857300-259) [n:127.0.0.1:32845_ c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:32845_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1660
   [junit4]   2> 89286 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001 of type NRT
   [junit4]   2> 89304 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 89371 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 89371 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 89371 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 89383 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58fe7039{/,null,AVAILABLE}
   [junit4]   2> 89383 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@12b92db2{HTTP/1.1,[http/1.1]}{127.0.0.1:35320}
   [junit4]   2> 89383 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server Started @89550ms
   [junit4]   2> 89383 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=35320, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001/cores}
   [junit4]   2> 89384 ERROR (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 89384 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 89384 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 89385 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 89385 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 89385 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-12T22:00:43.265Z
   [junit4]   2> 89403 INFO  (OverseerCollectionConfigSetProcessor-72960040291008516-127.0.0.1:36329_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 89436 INFO  (zkConnectionManagerCallback-62-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 89454 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 89454 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001/solr.xml
   [junit4]   2> 89458 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 89458 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 89460 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 89595 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39598/solr
   [junit4]   2> 89621 INFO  (zkConnectionManagerCallback-66-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 89637 INFO  (zkConnectionManagerCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 89652 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 89657 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 89660 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 89660 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35320_
   [junit4]   2> 89661 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89661 INFO  (zkCallback-51-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89667 INFO  (zkCallback-25-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89667 INFO  (zkCallback-32-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89667 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89675 INFO  (zkCallback-67-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 89760 INFO  (zkConnectionManagerCallback-75-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 89767 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 89769 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 89776 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 89801 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 89812 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 89812 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 89815 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35320_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001/cores
   [junit4]   2> 89961 INFO  (qtp1104857300-255) [n:127.0.0.1:32845_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:35320_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 90007 INFO  (OverseerThreadFactory-120-thread-4) [ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:35320_ for creating new replica
   [junit4]   2> 90032 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_    x:collection1_shard1_replica_n43] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 91049 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 91084 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 91253 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 91308 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 91312 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 91313 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 91313 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 91318 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4966378496681013]
   [junit4]   2> 91358 WARN  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 91499 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 91499 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 91500 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 91500 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 91502 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6029046529292041]
   [junit4]   2> 91503 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@6a479659[collection1_shard1_replica_n43] main]
   [junit4]   2> 91505 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 91505 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 91506 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 91506 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605823727283470336
   [junit4]   2> 91508 INFO  (searcherExecutor-153-thread-1-processing-n:127.0.0.1:35320_ x:collection1_shard1_replica_n43 c:collection1 s:shard1) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@6a479659[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 91525 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node44=0}, version=0}
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35320/collection1_shard1_replica_n43/
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:35320/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 91528 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 91533 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35320/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 91684 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 91689 INFO  (qtp1315168828-296) [n:127.0.0.1:35320_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1657
   [junit4]   2> 91692 INFO  (qtp1104857300-255) [n:127.0.0.1:32845_ c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:35320_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1731
   [junit4]   2> 91828 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-3-001 of type NRT
   [junit4]   2> 91828 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 91836 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 91836 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 91836 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 91839 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ef0f039{/,null,AVAILABLE}
   [junit4]   2> 91840 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@499d329f{HTTP/1.1,[http/1.1]}{127.0.0.1:35323}
   [junit4]   2> 91840 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.e.j.s.Server Started @92006ms
   [junit4]   2> 91840 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=35323, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-3-001/cores}
   [junit4]   2> 91840 ERROR (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 91841 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 91841 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 91841 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 91841 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 91841 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-12T22:00:45.721Z
   [junit4]   2> 91844 INFO  (zkConnectionManagerCallback-78-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 91845 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 91845 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-3-001/solr.xml
   [junit4]   2> 91850 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 91850 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 91851 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 91949 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39598/solr
   [junit4]   2> 91962 INFO  (zkConnectionManagerCallback-82-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 91967 INFO  (zkConnectionManagerCallback-84-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 91983 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 91992 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 91995 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 91995 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35323_
   [junit4]   2> 91999 INFO  (OverseerCollectionConfigSetProcessor-72960040291008516-127.0.0.1:36329_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 92007 INFO  (zkCallback-43-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-51-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-67-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-74-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-32-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92007 INFO  (zkCallback-25-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92015 INFO  (zkCallback-83-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 92052 INFO  (zkConnectionManagerCallback-91-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 92055 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 92056 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39598/solr ready
   [junit4]   2> 92057 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 92086 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 92106 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 92106 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e82b90f
   [junit4]   2> 92112 INFO  (TEST-ShardSplitTest.test-seed#[94FB101D78C6E874]) [n:127.0.0.1:35323_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_94FB101D78C6E874-001/shard-3-001/cores
   [junit4]   2> 92151 INFO  (qtp1104857300-261) [n:127.0.0.1:32845_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:35323_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 92158 INFO  (OverseerThreadFactory-120-thread-5) [ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:35323_ for creating new replica
   [junit4]   2> 92162 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_    x:collection1_shard2_replica_n45] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 93200 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 93213 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
   [junit4]   2> 93285 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 93430 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
   [junit4]   2> 93431 INFO  (qtp1731404548-335) [n:127.0.0.1:35323_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'so

[...truncated too long message...]

BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null273774251
     [copy] Copying 241 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null273774251
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null273774251

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 21ms :: artifacts dl 1ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 217 minutes 38 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 94 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/94/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([9B4022F728107B7F:3CAF3F54E75DF467]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:309)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([9B4022F728107B7F:1067F1266916D0FB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([9B4022F728107B7F:C6083C7EE7D6DD30]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12455 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_9B4022F728107B7F-001/init-core-data-001
   [junit4]   2> 530311 WARN  (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=11 numCloses=11
   [junit4]   2> 530311 INFO  (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 530333 INFO  (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 530335 INFO  (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 530603 DEBUG (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 530608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 530616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 530617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 530617 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 530650 INFO  (SUITE-TestTriggerIntegration-seed#[9B4022F728107B7F]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 530661 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
   [junit4]   2> 530661 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 530662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 530663 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 530683 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 530684 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2034) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 530712 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 530712 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 530712 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 530712 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 530713 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 530713 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 530727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 530727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 530741 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 530741 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 530753 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 530765 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 530766 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 530766 DEBUG (ScheduledTrigger-1691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530799 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530820 DEBUG (ScheduledTrigger-1691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530840 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530856 DEBUG (simCloudManagerPool-1690-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 530856 DEBUG (simCloudManagerPool-1690-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 530859 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 530860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 530860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 530860 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530880 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530901 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530924 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530934 DEBUG (simCloudManagerPool-1690-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"collection1"}}
   [junit4]   2> 530944 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530957 DEBUG (simCloudManagerPool-1690-thread-6) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 530958 DEBUG (simCloudManagerPool-1690-thread-6) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 530965 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 530976 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 530977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 530977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 530988 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531008 DEBUG (ScheduledTrigger-1691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531017 DEBUG (simCloudManagerPool-1690-thread-7) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531020 DEBUG (simCloudManagerPool-1690-thread-7) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 531020 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 531021 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 531021 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 531029 DEBUG (ScheduledTrigger-1691-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531050 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531070 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531090 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531111 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531127 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 531134 DEBUG (ScheduledTrigger-1691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
   [junit4]   2> 531183 DEBUG (simCloudManagerPool-1690-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531205 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":31583037236619116,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 531205 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":31583037236619116,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute]}
   [junit4]   2> 531206 INFO  (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 531206 INFO  (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 531207 INFO  (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 531207 INFO  (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 31583040864 sessionWrapper.createTime 31583040828800866, this.sessionWrapper.createTime 31583040828800866 
   [junit4]   2> 531208 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "__start__":3,
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":31583037236619116,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@35e4926e, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@32612b20, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@6405ac41], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
   [junit4]   2> 531212 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 531225 INFO  (simCloudManagerPool-1690-thread-11) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
   [junit4]   2> 531346 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 531355 INFO  (simCloudManagerPool-1690-thread-15) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
   [junit4]   2> 531395 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 531399 INFO  (simCloudManagerPool-1690-thread-19) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica
   [junit4]   2> 531423 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
   [junit4]   2> 531440 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[collection1, .system]
   [junit4]   2> ## Max replicas per node:	3
   [junit4]   2> ## Min replicas per node:	3
   [junit4]   2> ## Total replicas:		6
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ## * collection1		5
   [junit4]   2> ##		- active           5
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- ADDREPLICA         3
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- DELETESTATUS       2
   [junit4]   2> ##		- REQUESTSTATUS      2
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: search_rate_trigger
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=9B4022F728107B7F -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=is-IS -Dtests.timezone=Asia/Makassar -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.81s J0 | TestTriggerIntegration.testSearchRate <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9B4022F728107B7F:C6083C7EE7D6DD30]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 531471 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
   [junit4]   2> 531471 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 531471 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 531471 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 531471 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 531471 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 531471 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 531472 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 531472 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 531477 ERROR (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Unexpected Exception while querying status of requestId=search_rate_trigger/703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3/0
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:136) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	... 9 more
   [junit4]   2> 531480 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 531521 DEBUG (simCloudManagerPool-1690-thread-23) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531522 WARN  (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> java.lang.Exception: Error executing action: execute for trigger event: {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "__start__":3,
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":31583037236619116,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
   [junit4]   2>   "id":"703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":31583032700482116,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "__start__":3,
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":31583037236619116,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:120) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unexpected exception executing operation: action=ADDREPLICA&async=search_rate_trigger/703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3/0&waitForFinalState=true&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:110) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> Caused by: java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:136) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:136) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:85) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> 531523 DEBUG (AutoscalingActionExecutor-1692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 15919 ms for event id=703499d50ab244T13m7o5cf7ewbiseo1lezf3xu3
   [junit4]   2> 531523 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 531523 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 531523 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 531526 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 531527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 531528 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 531528 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 531540 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 531547 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 531547 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 531548 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531568 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531589 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531609 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531629 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531693 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531701 DEBUG (simCloudManagerPool-1693-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531701 DEBUG (simCloudManagerPool-1693-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 531704 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 531704 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 531704 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 531713 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531717 DEBUG (simCloudManagerPool-1693-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531730 DEBUG (simCloudManagerPool-1693-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 531733 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 531734 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 531734 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 531734 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
   [junit4]   2> 531734 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 531734 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 531734 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 531768 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 531768 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 531768 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 531769 DEBUG (simCloudManagerPool-1693-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531769 DEBUG (simCloudManagerPool-1693-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 531772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 531772 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 531772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 531772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
   [junit4]   2> 531773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 531773 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 531773 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 31583069938523616
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [31583069938523616], now=31583069940228516
   [junit4]   2> 531788 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7034a2809a1de0T13m7o5cf7ewbiseo1lezf3xu6",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31583069938523616,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583069938523616],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 531793 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
   [junit4]   2> 531801 DEBUG (simCloudManagerPool-1693-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531801 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> 531802 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7034a2809a1de0T13m7o5cf7ewbiseo1lezf3xu6",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31583069938523616,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583069938523616],
   [junit4]   2>     "_enqueue_time_":31583070169307466,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 531802 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger1
   [junit4]   2> 531822 DEBUG (simCloudManagerPool-1693-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531832 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 531832 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 531832 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 531832 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1544 ms for event id=7034a2809a1de0T13m7o5cf7ewbiseo1lezf3xu6
   [junit4]   2> 531932 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 531932 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 531932 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 31583077180246616
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [31583077180246616], now=31583077182267466
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7034a4303e0a58T13m7o5cf7ewbiseo1lezf3xu9",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":31583077180246616,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583077180246616],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 531933 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger2, node_added_trigger1]
   [junit4]   2> 531947 DEBUG (simCloudManagerPool-1693-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531947 DEBUG (ScheduledTrigger-1694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> 531948 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7034a4303e0a58T13m7o5cf7ewbiseo1lezf3xu9",
   [junit4]   2>   "source":"node_added_trigger2",
   [junit4]   2>   "eventTime":31583077180246616,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583077180246616],
   [junit4]   2>     "_enqueue_time_":31583077197169416,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 531948 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration last action at 31583070613850266 time = 31583077913931066
   [junit4]   2> 531948 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_added_trigger2
   [junit4]   2> 531962 DEBUG (simCloudManagerPool-1693-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 531962 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 531962 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 531962 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 531962 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 730 ms for event id=7034a4303e0a58T13m7o5cf7ewbiseo1lezf3xu9
   [junit4]   2> 531974 DEBUG (simCloudManagerPool-1693-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531974 DEBUG (simCloudManagerPool-1693-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 531975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 531979 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 531979 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 531979 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 531979 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
   [junit4]   2> 531981 DEBUG (simCloudManagerPool-1693-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 531982 DEBUG (simCloudManagerPool-1693-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 531982 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 531983 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 531983 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger2 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 531983 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 531983 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
   [junit4]   2> 532000 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
   [junit4]   2> 532000 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10002_solr
   [junit4]   2> 532000 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10002_solr]
   [junit4]   2> 532000 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7034a4f8833fecT13m7o5cf7ewbiseo1lezf3xuc",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":31583080540225516,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583080540225516],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 532000 DEBUG (ScheduledTrigger-1694-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
   [junit4]   2> 532046 DEBUG (simCloudManagerPool-1693-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 532046 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7034a4f8833fecT13m7o5cf7ewbiseo1lezf3xuc",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":31583080540225516,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583080540225516],
   [junit4]   2>     "_enqueue_time_":31583080555702616,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 532046 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger1
   [junit4]   2> 532058 DEBUG (simCloudManagerPool-1693-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 100ms
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 100ms
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 532058 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 591 ms for event id=7034a4f8833fecT13m7o5cf7ewbiseo1lezf3xuc
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10002_solr
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 532158 DEBUG (ScheduledTrigger-1694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 532159 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
   [junit4]   2> 532159 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10002_solr
   [junit4]   2> 532159 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10002_solr]
   [junit4]   2> 532159 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7034a6d0ed4a34T13m7o5cf7ewbiseo1lezf3xuf",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":31583088466020916,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583088466020916],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 532159 DEBUG (ScheduledTrigger-1694-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, .auto_add_replicas, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
   [junit4]   2> 532209 DEBUG (simCloudManagerPool-1693-thread-12) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 532209 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7034a6d0ed4a34T13m7o5cf7ewbiseo1lezf3xuf",
   [junit4]   2>   "source":"node_lost_trigger2",
   [junit4]   2>   "eventTime":31583088466020916,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31583088466020916],
   [junit4]   2>     "_enqueue_time_":31583088478294716,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 532210 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration last action at 31583082845917816 time = 31583091005117966
   [junit4]   2> 532210 INFO  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration action executed from node_lost_trigger2
   [junit4]   2> 532210 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
   [junit4]   2> 532210 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	1
   [junit4]   2> ## Dead nodes:		1
   [junit4]   2> ##		127.0.0.1:10002_solr
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        5
   [junit4]   2> ##		- update             7
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger1
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> ## * Trigger: node_added_trigger2
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> ## * Trigger: node_lost_trigger1
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> ## * Trigger: node_lost_trigger2
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> ## * Trigger: search_rate_trigger
   [junit4]   2> ##		- FAILED          1
   [junit4]   2> 
   [junit4]   2> 532237 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
   [junit4]   2> 532237 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 532238 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10002_solr
   [junit4]   2> 532238 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 532238 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 532238 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 532238 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 532238 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 532238 WARN  (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:756) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:321) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	... 10 more
   [junit4]   2> 532239 DEBUG (AutoscalingActionExecutor-1695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1486 ms for event id=7034a6d0ed4a34T13m7o5cf7ewbiseo1lezf3xuf
   [junit4]   2> 532239 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 532239 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 532239 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[9B4022F728107B7F]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 532251 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 532251 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 532252 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion -1
   [junit4]   2> 532252 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 532384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 532391 DEBUG (simCloudManagerPool-1696-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 532392 DEBUG (simCloudManagerPool-1696-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 532400 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 532401 DEBUG (ScheduledTrigger-1697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 532405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 14
   [junit4]   2> 532405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 532406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 532406 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 532417 INFO  (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 532417 DEBUG (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 532417 DEBUG (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 532418 DEBUG (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
   [junit4]   2> 532418 INFO  (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 532418 DEBUG (simCloudManagerPool-1696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 532418 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
   [junit4]   2> 532419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigge

[...truncated too long message...]

s-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2078711012
     [copy] Copying 241 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2078711012
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2078711012

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 65ms :: artifacts dl 1ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 173 minutes 21 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 93 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/93/

11 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
distrib-dup-test-chain-explicit: doc#3 has wrong value for regex_dup_A_s expected:<x_x> but was:<null>

Stack Trace:
java.lang.AssertionError: distrib-dup-test-chain-explicit: doc#3 has wrong value for regex_dup_A_s expected:<x_x> but was:<null>
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15:CE550B644D67E3ED]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:704)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:381)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)


FAILED:  org.apache.solr.cloud.TestPullReplica.testKillLeader

Error Message:
Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15:F17C00A81201A43]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.TestPullReplica.waitForNumDocsInAllReplicas(TestPullReplica.java:542)
	at org.apache.solr.cloud.TestPullReplica.doTestNoLeader(TestPullReplica.java:490)
	at org.apache.solr.cloud.TestPullReplica.testKillLeader(TestPullReplica.java:309)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplica.testRemoveAllWriterReplicas

Error Message:
Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15:D50FA885F76567BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.TestPullReplica.waitForNumDocsInAllReplicas(TestPullReplica.java:542)
	at org.apache.solr.cloud.TestPullReplica.doTestNoLeader(TestPullReplica.java:490)
	at org.apache.solr.cloud.TestPullReplica.testRemoveAllWriterReplicas(TestPullReplica.java:303)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.component.InfixSuggestersTest.testBlendedInfixSuggesterBuildThenReload

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.InfixSuggestersTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)


FAILED:  org.apache.solr.client.solrj.embedded.TestSolrProperties.testProperties

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([38EAB06EC44A11A7]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.TestSolrProperties

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([38EAB06EC44A11A7]:0)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([460134BEE39B8E15]:0)




Build Log:
[...truncated 12217 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplica
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/init-core-data-001
   [junit4]   2> 29184 WARN  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 29184 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 29186 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 29186 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 29190 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001
   [junit4]   2> 29234 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 29236 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 29237 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 29536 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46364
   [junit4]   2> 29617 ERROR (Thread-2) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 29847 INFO  (zkConnectionManagerCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30007 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1074e029a930000, likely client has closed socket
   [junit4]   2> 30782 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 30796 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 30904 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 30904 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 30909 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 31045 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 31045 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 31045 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 31054 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25244da4{/solr,null,AVAILABLE}
   [junit4]   2> 31067 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e5ef0d3{/solr,null,AVAILABLE}
   [junit4]   2> 31139 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@51d53c93{HTTP/1.1,[http/1.1]}{127.0.0.1:50507}
   [junit4]   2> 31140 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server Started @31315ms
   [junit4]   2> 31142 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@17c4e64d{HTTP/1.1,[http/1.1]}{127.0.0.1:45633}
   [junit4]   2> 31142 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server Started @31318ms
   [junit4]   2> 31143 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45633}
   [junit4]   2> 31148 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50507}
   [junit4]   2> 31241 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
   [junit4]   2> 31252 ERROR (jetty-launcher-3-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 31252 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 31252 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 31253 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 31254 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 31255 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-09T07:34:29.907Z
   [junit4]   2> 31253 ERROR (jetty-launcher-3-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 31256 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 31256 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 31256 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 31256 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 31257 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 31257 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-09T07:34:29.909Z
   [junit4]   2> 31353 INFO  (zkConnectionManagerCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31396 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 31408 INFO  (zkConnectionManagerCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31422 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 32364 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46364/solr
   [junit4]   2> 32556 INFO  (zkConnectionManagerCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32650 INFO  (zkConnectionManagerCallback-16-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33716 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46364/solr
   [junit4]   2> 33765 INFO  (zkConnectionManagerCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33809 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 33814 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45633_solr
   [junit4]   2> 33815 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.Overseer Overseer (id=74113691965128708-127.0.0.1:45633_solr-n_0000000000) starting
   [junit4]   2> 33859 INFO  (zkConnectionManagerCallback-24-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33879 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 33882 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50507_solr
   [junit4]   2> 34028 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 34029 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 34133 INFO  (zkConnectionManagerCallback-31-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 34135 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 34172 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46364/solr ready
   [junit4]   2> 34452 INFO  (zkConnectionManagerCallback-36-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 34465 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 34473 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46364/solr ready
   [junit4]   2> 34492 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 34886 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 34886 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45633_solr
   [junit4]   2> 34897 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 34930 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 34946 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50507.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 34947 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50507.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 34947 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50507.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 34970 INFO  (zkCallback-30-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 34975 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:50507_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node1/.
   [junit4]   2> 35013 INFO  (zkCallback-35-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 35106 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 35498 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45633.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 35576 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45633.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 35605 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45633.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 35607 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:45633_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node2/.
   [junit4]   2> 35992 INFO  (zkConnectionManagerCallback-40-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 36060 INFO  (zkConnectionManagerCallback-45-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 36071 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 36073 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46364/solr ready
   [junit4]   2> 36082 INFO  (SUITE-TestPullReplica-seed#[460134BEE39B8E15]-worker) [    ] o.a.s.c.TestPullReplica Using legacyCloud?: false
   [junit4]   2> 37024 INFO  (qtp1795463531-43) [n:127.0.0.1:50507_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 37057 INFO  (qtp1795463531-43) [n:127.0.0.1:50507_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=47
   [junit4]   2> 37226 INFO  (TEST-TestPullReplica.testAddDocs-seed#[460134BEE39B8E15]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testAddDocs
   [junit4]   2> 37259 INFO  (qtp1795463531-41) [n:127.0.0.1:50507_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=3&collection.configName=conf&maxShardsPerNode=100&name=pull_replica_test_add_docs&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 37300 INFO  (OverseerThreadFactory-32-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection pull_replica_test_add_docs
   [junit4]   2> 37324 WARN  (OverseerThreadFactory-32-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection pull_replica_test_add_docs is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 37537 INFO  (OverseerStateUpdate-74113691965128708-127.0.0.1:45633_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_test_add_docs",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_test_add_docs_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:45633/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 37559 INFO  (OverseerStateUpdate-74113691965128708-127.0.0.1:45633_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_test_add_docs",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_test_add_docs_shard1_replica_p2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:50507/solr",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 37563 INFO  (OverseerStateUpdate-74113691965128708-127.0.0.1:45633_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_test_add_docs",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_test_add_docs_shard1_replica_p4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:45633/solr",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 37566 INFO  (OverseerStateUpdate-74113691965128708-127.0.0.1:45633_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_test_add_docs",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_test_add_docs_shard1_replica_p5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:50507/solr",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 37825 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr    x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p2&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 37826 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr    x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p5&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 37835 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr    x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 37846 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr    x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_n1&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 37848 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr    x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p4&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 38921 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 38921 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 38957 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 39017 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 39043 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.s.IndexSchema [pull_replica_test_add_docs_shard1_replica_p5] Schema name=minimal
   [junit4]   2> 39051 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.s.IndexSchema [pull_replica_test_add_docs_shard1_replica_p4] Schema name=minimal
   [junit4]   2> 39053 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.s.IndexSchema [pull_replica_test_add_docs_shard1_replica_p2] Schema name=minimal
   [junit4]   2> 39087 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 39088 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_add_docs_shard1_replica_p4' using configuration from collection pull_replica_test_add_docs, trusted=true
   [junit4]   2> 39088 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.s.IndexSchema [pull_replica_test_add_docs_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 39096 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 39101 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_add_docs_shard1_replica_p2' using configuration from collection pull_replica_test_add_docs, trusted=true
   [junit4]   2> 39120 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 39121 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_add_docs_shard1_replica_n1' using configuration from collection pull_replica_test_add_docs, trusted=true
   [junit4]   2> 39123 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45633.solr.core.pull_replica_test_add_docs.shard1.replica_n1' (registry 'solr.core.pull_replica_test_add_docs.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 39123 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 39123 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SolrCore [[pull_replica_test_add_docs_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node2/pull_replica_test_add_docs_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node2/./pull_replica_test_add_docs_shard1_replica_n1/data/]
   [junit4]   2> 39124 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45633.solr.core.pull_replica_test_add_docs.shard1.replica_p4' (registry 'solr.core.pull_replica_test_add_docs.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 39124 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 39125 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.SolrCore [[pull_replica_test_add_docs_shard1_replica_p4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node2/pull_replica_test_add_docs_shard1_replica_p4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node2/./pull_replica_test_add_docs_shard1_replica_p4/data/]
   [junit4]   2> 39126 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50507.solr.core.pull_replica_test_add_docs.shard1.replica_p2' (registry 'solr.core.pull_replica_test_add_docs.shard1.replica_p2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 39133 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 39133 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_add_docs_shard1_replica_p5' using configuration from collection pull_replica_test_add_docs, trusted=true
   [junit4]   2> 39134 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50507.solr.core.pull_replica_test_add_docs.shard1.replica_p5' (registry 'solr.core.pull_replica_test_add_docs.shard1.replica_p5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6356814b
   [junit4]   2> 39134 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 39134 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.SolrCore [[pull_replica_test_add_docs_shard1_replica_p5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node1/pull_replica_test_add_docs_shard1_replica_p5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node1/./pull_replica_test_add_docs_shard1_replica_p5/data/]
   [junit4]   2> 39149 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 39149 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.SolrCore [[pull_replica_test_add_docs_shard1_replica_p2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node1/pull_replica_test_add_docs_shard1_replica_p2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_460134BEE39B8E15-001/tempDir-001/node1/./pull_replica_test_add_docs_shard1_replica_p2/data/]
   [junit4]   2> 39656 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 39657 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 39673 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7adaa573[pull_replica_test_add_docs_shard1_replica_p2] main]
   [junit4]   2> 39682 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 39683 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 39684 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 39738 INFO  (searcherExecutor-39-thread-1-processing-n:127.0.0.1:50507_solr x:pull_replica_test_add_docs_shard1_replica_p2 c:pull_replica_test_add_docs s:shard1 r:core_node6) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p2] Registered new searcher Searcher@7adaa573[pull_replica_test_add_docs_shard1_replica_p2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 39817 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.ZkController pull_replica_test_add_docs_shard1_replica_p2 starting background replication from leader
   [junit4]   2> 39819 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 39883 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 39883 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 39988 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 39989 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 39997 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a32c0a3[pull_replica_test_add_docs_shard1_replica_p4] main]
   [junit4]   2> 40000 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 40000 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 40003 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40010 INFO  (searcherExecutor-38-thread-1-processing-n:127.0.0.1:45633_solr x:pull_replica_test_add_docs_shard1_replica_p4 c:pull_replica_test_add_docs s:shard1 r:core_node7) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p4] Registered new searcher Searcher@4a32c0a3[pull_replica_test_add_docs_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 40015 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.ZkController pull_replica_test_add_docs_shard1_replica_p4 starting background replication from leader
   [junit4]   2> 40016 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 40020 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 40021 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40036 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 40036 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 40046 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.s.SolrIndexSearcher Opening [Searcher@fa17bde[pull_replica_test_add_docs_shard1_replica_p5] main]
   [junit4]   2> 40061 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 40062 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 40063 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40071 INFO  (searcherExecutor-41-thread-1-processing-n:127.0.0.1:50507_solr x:pull_replica_test_add_docs_shard1_replica_p5 c:pull_replica_test_add_docs s:shard1 r:core_node8) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p5] Registered new searcher Searcher@fa17bde[pull_replica_test_add_docs_shard1_replica_p5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 40098 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.ZkController pull_replica_test_add_docs_shard1_replica_p5 starting background replication from leader
   [junit4]   2> 40098 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 40101 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 40101 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40184 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 40184 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 40258 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 40258 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 40281 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@505c97cd[pull_replica_test_add_docs_shard1_replica_n1] main]
   [junit4]   2> 40283 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 40283 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 40284 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40286 INFO  (searcherExecutor-40-thread-1-processing-n:127.0.0.1:45633_solr x:pull_replica_test_add_docs_shard1_replica_n1 c:pull_replica_test_add_docs s:shard1 r:core_node3) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_n1] Registered new searcher Searcher@505c97cd[pull_replica_test_add_docs_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 40287 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605497445615140864
   [junit4]   2> 40307 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/pull_replica_test_add_docs/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 40360 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 40360 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 40360 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/
   [junit4]   2> 40362 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 40362 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/ has no replicas
   [junit4]   2> 40362 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 40565 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/ shard1
   [junit4]   2> 40676 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 40682 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Replica core_node3 is leader but it's state is down, skipping replication
   [junit4]   2> 40683 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Replica core_node3 is leader but it's state is down, skipping replication
   [junit4]   2> 40688 INFO  (qtp1225175455-53) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_n1&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2842
   [junit4]   2> 40781 INFO  (zkCallback-15-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_add_docs/state.json] for collection [pull_replica_test_add_docs] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 40889 INFO  (qtp1795463531-39) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p2&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=3068
   [junit4]   2> 41010 INFO  (zkCallback-23-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_add_docs/state.json] for collection [pull_replica_test_add_docs] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 41011 INFO  (zkCallback-15-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_add_docs/state.json] for collection [pull_replica_test_add_docs] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 41040 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node7 x:pull_replica_test_add_docs_shard1_replica_p4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p4&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=3191
   [junit4]   2> 41106 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node8 x:pull_replica_test_add_docs_shard1_replica_p5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=pull_replica_test_add_docs_shard1_replica_p5&action=CREATE&numShards=1&collection=pull_replica_test_add_docs&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=3282
   [junit4]   2> 41125 INFO  (OverseerCollectionConfigSetProcessor-74113691965128708-127.0.0.1:45633_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 41128 INFO  (qtp1795463531-41) [n:127.0.0.1:50507_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 41206 INFO  (zkCallback-23-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_add_docs/state.json] for collection [pull_replica_test_add_docs] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 41208 INFO  (zkCallback-15-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_add_docs/state.json] for collection [pull_replica_test_add_docs] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 42129 INFO  (qtp1795463531-41) [n:127.0.0.1:50507_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=3&collection.configName=conf&maxShardsPerNode=100&name=pull_replica_test_add_docs&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=4869
   [junit4]   2> 42433 INFO  (qtp1225175455-49) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/pull_replica_test_add_docs/terms/shard1 to Terms{values={core_node3=1}, version=1}
   [junit4]   2> 42434 INFO  (qtp1225175455-49) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1605497447714390016)]} 0 208
   [junit4]   2> 42438 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/
   [junit4]   2> 42458 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1605497447891599360,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 42458 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@15e0d8f1 commitCommandVersion:1605497447891599360
   [junit4]   2> 42480 INFO  (qtp1225175455-52) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=39
   [junit4]   2> 42481 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 42481 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 42481 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 42481 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 42481 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 42485 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 42485 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@53e1aae9 commitCommandVersion:0
   [junit4]   2> 42933 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
   [junit4]   2> 42933 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/
   [junit4]   2> 42942 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 42945 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 42945 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 42945 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 42945 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 42945 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 42948 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d6cda03[pull_replica_test_add_docs_shard1_replica_p5] main]
   [junit4]   2> 42949 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 42951 INFO  (searcherExecutor-41-thread-1) [    ] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p5] Registered new searcher Searcher@7d6cda03[pull_replica_test_add_docs_shard1_replica_p5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 42957 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 42957 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@76932e16 commitCommandVersion:0
   [junit4]   2> 42965 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3969d48f[pull_replica_test_add_docs_shard1_replica_p2] main]
   [junit4]   2> 42968 INFO  (searcherExecutor-39-thread-1) [    ] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p2] Registered new searcher Searcher@3969d48f[pull_replica_test_add_docs_shard1_replica_p2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 42968 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 43017 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
   [junit4]   2> 43017 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:45633/solr/pull_replica_test_add_docs_shard1_replica_n1/
   [junit4]   2> 43029 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 43031 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 43031 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 43031 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 43031 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 43031 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 43032 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43032 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6ba5856b commitCommandVersion:0
   [junit4]   2> 43035 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c310a6b[pull_replica_test_add_docs_shard1_replica_p4] main]
   [junit4]   2> 43036 INFO  (indexFetcher-53-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 43038 INFO  (searcherExecutor-38-thread-1) [    ] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_p4] Registered new searcher Searcher@5c310a6b[pull_replica_test_add_docs_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 43112 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ae8fd56[pull_replica_test_add_docs_shard1_replica_n1] main]
   [junit4]   2> 43115 INFO  (searcherExecutor-40-thread-1-processing-n:127.0.0.1:45633_solr x:pull_replica_test_add_docs_shard1_replica_n1 c:pull_replica_test_add_docs s:shard1 r:core_node3) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_add_docs_shard1_replica_n1] Registered new searcher Searcher@ae8fd56[pull_replica_test_add_docs_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C1)))}
   [junit4]   2> 43119 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 43119 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=pull_replica_test_add_docs:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 662
   [junit4]   2> 43143 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=3
   [junit4]   2> 43175 INFO  (qtp1795463531-42) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=15
   [junit4]   2> 43279 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43384 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43498 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43602 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43705 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43809 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 43919 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44085 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44188 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44296 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44399 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44530 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44635 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44747 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 44852 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45046 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45149 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45254 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45357 INFO  (qtp1795463531-46) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45440 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 45445 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 45445 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 1531121681111
   [junit4]   2> 45445 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 2
   [junit4]   2> 45445 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 1531121681258
   [junit4]   2> 45445 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 45574 INFO  (qtp1795463531-42) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45687 INFO  (qtp1795463531-42) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45457 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=10
   [junit4]   2> 45730 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Number of files in latest index in master: 18
   [junit4]   2> 45732 INFO  (indexFetcher-57-thread-1) [    ] o.a.s.h.IndexFetcher Starting download (fullCopy=true) to MockDirectoryWrapper(RAMDirectory@76bd41cf lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3e59cc25)
   [junit4]   2> 45760 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=5
   [junit4]   2> 45796 INFO  (qtp1795463531-42) [n:127.0.0.1:50507_solr c:pull_replica_test_add_docs s:shard1 r:core_node6 x:pull_replica_test_add_docs_shard1_replica_p2] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_p2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 45828 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneVarGapFixedInterval_0.tiv&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45839 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45846 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_MockRandom_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45850 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneVarGapFixedInterval_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45852 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45857 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneVarGapFixedInterval_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45860 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45863 INFO  (qtp1225175455-52) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 45864 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 45864 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 1531121681111
   [junit4]   2> 45864 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 2
   [junit4]   2> 45864 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 1531121681609
   [junit4]   2> 45864 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 45866 INFO  (qtp1225175455-52) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=1
   [junit4]   2> 45867 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Number of files in latest index in master: 18
   [junit4]   2> 45867 INFO  (indexFetcher-48-thread-1) [    ] o.a.s.h.IndexFetcher Starting download (fullCopy=true) to MockDirectoryWrapper(RAMDirectory@3578337b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5d08163f)
   [junit4]   2> 45869 INFO  (qtp1225175455-48) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45886 INFO  (qtp1225175455-55) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneVarGapFixedInterval_0.tiv&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45889 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.dii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45892 INFO  (qtp1225175455-52) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_docs_shard1_replica_n1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 45894 INFO  (qtp1225175455-54) [n:127.0.0.1:45633_solr c:pull_replica_test_add_docs s:shard1 r:core_node3 x:pull_replica_test_add_docs_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_add_d

[...truncated too long message...]

jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null23856135
     [copy] Copying 241 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null23856135
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null23856135

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 97ms :: artifacts dl 6ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 407 minutes 20 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 92 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/92/

5 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:56043/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:60610/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:56043/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:60610/solr/MoveReplicaHDFSTest_failed_coll_true]
	at __randomizedtesting.SeedInfo.seed([39F0A905F9864303:933D7AF74E5596D3]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
	at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:290)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:60610/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
	... 46 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=31421621231216874, stage=STARTED, actionName='null', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622466404024, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622521019174, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622537999724, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421630983363574, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421631007373024, stage=SUCCEEDED, actionName='null', event={   "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",   "source":"index_size_trigger2",   "eventTime":31421617682565324,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":31421621191990274,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639276065924, stage=STARTED, actionName='null', event={   "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",   "source":"index_size_trigger2",   "eventTime":31421636104751724,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31421639262896874,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639418366074, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",   "source":"index_size_trigger2",   "eventTime":31421636104751724,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31421639262896874,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639575558924, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",   "source":"index_size_trigger2",   "eventTime":31421636104751724,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31421639262896874,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639837387874, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",   "source":"index_size_trigger2",   "eventTime":31421636104751724,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":31421639262896874,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=31421621231216874, stage=STARTED, actionName='null', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622466404024, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622521019174, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421622537999724, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421630983363574, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421631007373024, stage=SUCCEEDED, actionName='null', event={
  "id":"6fa1cb783cc8ccT3nuqvrvouq1q93q2vifa3wlr8",
  "source":"index_size_trigger2",
  "eventTime":31421617682565324,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":31421621191990274,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639276065924, stage=STARTED, actionName='null', event={
  "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",
  "source":"index_size_trigger2",
  "eventTime":31421636104751724,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31421639262896874,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639418366074, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",
  "source":"index_size_trigger2",
  "eventTime":31421636104751724,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31421639262896874,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639575558924, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",
  "source":"index_size_trigger2",
  "eventTime":31421636104751724,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31421639262896874,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=31421639837387874, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6fa1cfc2490a6cT3nuqvrvouq1q93q2vifa3wlrb",
  "source":"index_size_trigger2",
  "eventTime":31421636104751724,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"base_url\":\"http://127.0.0.1:10032/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"base_url\":\"http://127.0.0.1:10033/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10033_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":31421639262896874,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}] expected:<6> but was:<10>
	at __randomizedtesting.SeedInfo.seed([39F0A905F9864303:7E1045D6798AFD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:317)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([39F0A905F9864303]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([39F0A905F9864303]:0)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([39F0A905F9864303:64B8B78C3640E54C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12629 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_39F0A905F9864303-001/init-core-data-001
   [junit4]   2> 755901 DEBUG (SUITE-TestTriggerIntegration-seed#[39F0A905F9864303]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10007_solr
   [junit4]   2> 755902 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 755902 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 755903 INFO  (SUITE-TestTriggerIntegration-seed#[39F0A905F9864303]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 755910 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 755910 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 755910 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 755917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 755917 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 755924 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 755924 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 755925 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 755927 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 755927 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 755928 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 755949 DEBUG (ScheduledTrigger-2992-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756086 DEBUG (ScheduledTrigger-2992-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756094 DEBUG (simCloudManagerPool-2991-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 756094 DEBUG (simCloudManagerPool-2991-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 756095 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 756095 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 756095 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 756106 DEBUG (ScheduledTrigger-2992-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756107 INFO  (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 756107 DEBUG (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 756107 DEBUG (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 756108 DEBUG (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 756108 INFO  (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 756108 DEBUG (simCloudManagerPool-2991-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 756108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 756108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 756108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 756108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 756108 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 756108 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 756127 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 756129 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 756129 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 756129 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10009_solr at time 31310810837722024
   [junit4]   2> 756151 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 756151 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 756151 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 756151 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10009_solr] added at times [31310810837722024], now=31310811970687224
   [junit4]   2> 756151 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"]}}
   [junit4]   2> 756152 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
   [junit4]   2> 756183 DEBUG (simCloudManagerPool-2991-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10009_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 756183 DEBUG (ScheduledTrigger-2992-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10009_solr
   [junit4]   2> 756184 DEBUG (AutoscalingActionExecutor-2993-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "_enqueue_time_":31310812002775224,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"]}}
   [junit4]   2> 756184 INFO  (AutoscalingActionExecutor-2993-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "_enqueue_time_":31310812002775224,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"]}}
   [junit4]   2> 756184 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 756184 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 756185 WARN  (AutoscalingActionExecutor-2993-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 756185 DEBUG (AutoscalingActionExecutor-2993-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 92 ms for event id=6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 756185 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 756186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 756197 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10008_solr
   [junit4]   2> 756197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 756198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 756198 DEBUG (ScheduledTrigger-2995-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756198 DEBUG (ScheduledTrigger-2995-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10008_solr
   [junit4]   2> 756198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 756198 DEBUG (ScheduledTrigger-2995-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "_dequeue_time_":31310814311686074,
   [junit4]   2>     "_enqueue_time_":31310812002775224,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"],
   [junit4]   2>     "replaying":true}}
   [junit4]   2> 756203 DEBUG (ScheduledTrigger-2995-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
   [junit4]   2> 756208 DEBUG (simCloudManagerPool-2994-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10009_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 756208 DEBUG (ScheduledTrigger-2995-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 756208 DEBUG (ScheduledTrigger-2995-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 756226 DEBUG (AutoscalingActionExecutor-2996-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "_dequeue_time_":31310814311686074,
   [junit4]   2>     "_enqueue_time_":31310812002775224,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"],
   [junit4]   2>     "replaying":true}}
   [junit4]   2> 756226 INFO  (AutoscalingActionExecutor-2996-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":31310810837722024,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[31310810837722024],
   [junit4]   2>     "_dequeue_time_":31310814311686074,
   [junit4]   2>     "_enqueue_time_":31310812002775224,
   [junit4]   2>     "nodeNames":["127.0.0.1:10009_solr"],
   [junit4]   2>     "replaying":true}}
   [junit4]   2> 756248 DEBUG (simCloudManagerPool-2994-thread-2) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10009_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 756248 DEBUG (AutoscalingActionExecutor-2996-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 756248 DEBUG (AutoscalingActionExecutor-2996-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
   [junit4]   2> 756248 DEBUG (AutoscalingActionExecutor-2996-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1092 ms for event id=6f3d043d9ca3a8T3nuqvrvouq1q93q2vifa3wclq
   [junit4]   2> 756287 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	1
   [junit4]   2> ## Dead nodes:		1
   [junit4]   2> ##		127.0.0.1:10008_solr
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        2
   [junit4]   2> ##		- update             3
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger1
   [junit4]   2> ##		- STARTED         2
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> 
   [junit4]   2> 756300 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 756300 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10008_solr
   [junit4]   2> 756300 INFO  (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 756300 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 756300 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 756300 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 756300 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 756301 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 756301 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 756301 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[39F0A905F9864303]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 756311 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 756312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 756312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
   [junit4]   2> 756312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 756315 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 756315 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 756315 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 756315 DEBUG (ScheduledTrigger-2998-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756336 DEBUG (ScheduledTrigger-2998-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756356 DEBUG (ScheduledTrigger-2998-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756376 DEBUG (ScheduledTrigger-2998-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756396 DEBUG (ScheduledTrigger-2998-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756412 DEBUG (simCloudManagerPool-2997-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 756412 DEBUG (simCloudManagerPool-2997-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 756413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 756413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 756413 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 756417 DEBUG (ScheduledTrigger-2998-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 756418 DEBUG (simCloudManagerPool-2997-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 756418 DEBUG (simCloudManagerPool-2997-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 756418 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 756419 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 756419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10009_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 756419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 756419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 756419 DEBUG (ScheduledTrigger-2998-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
   [junit4]   2> 756437 DEBUG (ScheduledTrigger-2998-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 756437 DEBUG (ScheduledTrigger-2998-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
   [junit4]   2> 756439 DEBUG (ScheduledTrigger-2998-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
   [junit4]   2> 756439 DEBUG (ScheduledTrigger-2998-thread-4) [    

[...truncated too long message...]

slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2015701526
     [copy] Copying 241 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2015701526
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null2015701526

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 109ms :: artifacts dl 16ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 255 minutes 22 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 91 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/91/

4 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at http://127.0.0.1:59685/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:59685/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([EC0E8466C13A747D:D1D62A4AF9D42A0D]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
	at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59685/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([EC0E8466C13A747D:8FC5B2E458F50750]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:59980/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:59980/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([EC0E8466C13A747D:4FF42AC346D29ED8]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:170)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:59980/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([EC0E8466C13A747D:4BE199C50E77FB65]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:309)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13824 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/init-core-data-001
   [junit4]   2> 2842735 WARN  (SUITE-TestAuthenticationFramework-seed#[EC0E8466C13A747D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=25 numCloses=25
   [junit4]   2> 2842735 INFO  (SUITE-TestAuthenticationFramework-seed#[EC0E8466C13A747D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2842737 INFO  (SUITE-TestAuthenticationFramework-seed#[EC0E8466C13A747D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2842737 INFO  (SUITE-TestAuthenticationFramework-seed#[EC0E8466C13A747D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2842739 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001
   [junit4]   2> 2842739 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2842740 INFO  (Thread-5824) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2842740 INFO  (Thread-5824) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2842742 ERROR (Thread-5824) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2842840 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.c.ZkTestServer start zk server on port:37962
   [junit4]   2> 2842842 INFO  (zkConnectionManagerCallback-4844-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2842853 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1073e50b7e50000, likely client has closed socket
   [junit4]   2> 2842854 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2842855 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 2842856 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e030d49{/solr,null,AVAILABLE}
   [junit4]   2> 2842857 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15bb59e2{/solr,null,AVAILABLE}
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@239c7431{HTTP/1.1,[http/1.1]}{127.0.0.1:52721}
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@51e483b6{HTTP/1.1,[http/1.1]}{127.0.0.1:59685}
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-2) [    ] o.e.j.s.Server Started @2843079ms
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-1) [    ] o.e.j.s.Server Started @2843079ms
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52721}
   [junit4]   2> 2842858 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59685}
   [junit4]   2> 2842859 ERROR (jetty-launcher-4841-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2842859 ERROR (jetty-launcher-4841-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2842859 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-06T06:25:51.498Z
   [junit4]   2> 2842864 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2842864 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2842864 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2842864 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-07-06T06:25:51.503Z
   [junit4]   2> 2842872 INFO  (zkConnectionManagerCallback-4846-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2842876 INFO  (zkConnectionManagerCallback-4848-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2842876 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2842877 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2843162 INFO  (jetty-launcher-4841-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37962/solr
   [junit4]   2> 2843163 INFO  (zkConnectionManagerCallback-4852-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843165 INFO  (zkConnectionManagerCallback-4854-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843211 INFO  (jetty-launcher-4841-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37962/solr
   [junit4]   2> 2843213 INFO  (zkConnectionManagerCallback-4860-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843216 INFO  (zkConnectionManagerCallback-4862-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843257 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2843258 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52721_solr
   [junit4]   2> 2843260 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.Overseer Overseer (id=74096435278446598-127.0.0.1:52721_solr-n_0000000000) starting
   [junit4]   2> 2843273 INFO  (zkConnectionManagerCallback-4869-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843276 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37962/solr ready
   [junit4]   2> 2843287 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2843288 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52721_solr
   [junit4]   2> 2843288 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59685_solr
   [junit4]   2> 2843289 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 2843292 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 2843311 INFO  (zkCallback-4861-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2843321 INFO  (zkCallback-4853-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2843325 INFO  (zkCallback-4868-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2843330 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2843354 INFO  (zkConnectionManagerCallback-4875-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843356 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2843357 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37962/solr ready
   [junit4]   2> 2843363 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2843378 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52721.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843397 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52721.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843397 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52721.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843399 INFO  (jetty-launcher-4841-thread-2) [n:127.0.0.1:52721_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node2/.
   [junit4]   2> 2843409 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59685.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843454 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59685.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843454 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59685.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2843456 INFO  (jetty-launcher-4841-thread-1) [n:127.0.0.1:59685_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node1/.
   [junit4]   2> 2843507 INFO  (zkConnectionManagerCallback-4878-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843511 INFO  (zkConnectionManagerCallback-4883-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2843513 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2843514 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37962/solr ready
   [junit4]   2> 2843517 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[EC0E8466C13A747D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 2843519 INFO  (qtp1514045924-19850) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=0a2874e6-04e3-4691-b1dd-586c6490b5e6&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2843523 INFO  (qtp1514045924-19850) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=0a2874e6-04e3-4691-b1dd-586c6490b5e6&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 2843536 INFO  (qtp1514045924-19842) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2843537 INFO  (qtp1514045924-19842) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2843544 INFO  (OverseerThreadFactory-7017-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 2843656 INFO  (OverseerStateUpdate-74096435278446598-127.0.0.1:52721_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:52721/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2843659 INFO  (OverseerStateUpdate-74096435278446598-127.0.0.1:52721_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59685/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2843662 INFO  (OverseerStateUpdate-74096435278446598-127.0.0.1:52721_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:52721/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2843666 INFO  (OverseerStateUpdate-74096435278446598-127.0.0.1:52721_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:59685/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2843888 INFO  (qtp1514045924-19848) [n:127.0.0.1:59685_solr    x:testcollection_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 2843889 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr    x:testcollection_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2843889 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr    x:testcollection_shard2_replica_n6] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2843891 INFO  (qtp1514045924-19841) [n:127.0.0.1:59685_solr    x:testcollection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=4
   [junit4]   2> 2843891 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr    x:testcollection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2843891 INFO  (qtp1795113159-19845) [n:127.0.0.1:52721_solr    x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=3
   [junit4]   2> 2843892 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr    x:testcollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2843892 INFO  (qtp1795113159-19845) [n:127.0.0.1:52721_solr    x:testcollection_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 2843892 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr    x:testcollection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2843896 INFO  (qtp1514045924-19852) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649
   [junit4]   2> 2843896 INFO  (qtp1514045924-19852) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2843897 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr    x:testcollection_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2844538 INFO  (qtp1514045924-19846) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2844539 INFO  (qtp1514045924-19846) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2844898 INFO  (qtp1514045924-19852) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649
   [junit4]   2> 2844898 INFO  (qtp1514045924-19852) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2844916 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2844920 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2844922 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2844928 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2844933 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2844937 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2844937 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2844937 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52721.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2844937 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2844938 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 2844941 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 2844942 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
   [junit4]   2> 2844944 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2844944 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2844944 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59685.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2844945 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2844945 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node1/testcollection_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node1/./testcollection_shard1_replica_n2/data/]
   [junit4]   2> 2844946 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema [testcollection_shard2_replica_n6] Schema name=minimal
   [junit4]   2> 2844982 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2844982 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n6' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59685.solr.core.testcollection.shard2.replica_n6' (registry 'solr.core.testcollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [[testcollection_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node1/testcollection_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node1/./testcollection_shard2_replica_n6/data/]
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52721.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@fdddafd
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2845016 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node2/testcollection_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestAuthenticationFramework_EC0E8466C13A747D-001/tempDir-001/node2/./testcollection_shard2_replica_n4/data/]
   [junit4]   2> 2845176 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2845176 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2845177 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2845178 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2845179 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bc406a[testcollection_shard1_replica_n1] main]
   [junit4]   2> 2845181 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2845181 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2845181 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2845181 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2845182 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2845182 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2845182 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2845188 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d423e51[testcollection_shard1_replica_n2] main]
   [junit4]   2> 2845190 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2845190 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2845191 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2845191 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2845191 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2845193 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2845193 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2845194 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@34e97194[testcollection_shard2_replica_n4] main]
   [junit4]   2> 2845196 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2845199 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2845200 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2845200 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605221329235083264
   [junit4]   2> 2845208 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0}, version=0}
   [junit4]   2> 2845218 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2845219 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605221329255006208
   [junit4]   2> 2845219 INFO  (searcherExecutor-7027-thread-1-processing-x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 r:core_node5 CREATE n:127.0.0.1:59685_solr c:testcollection s:shard1) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@6d423e51[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2845225 INFO  (searcherExecutor-7029-thread-1-processing-x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 r:core_node7 CREATE n:127.0.0.1:52721_solr c:testcollection s:shard2) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@34e97194[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2845229 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node5=0}, version=0}
   [junit4]   2> 2845234 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2845234 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605221329270734848
   [junit4]   2> 2845235 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2845235 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2845236 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2845236 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2845237 INFO  (searcherExecutor-7026-thread-1-processing-x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 r:core_node3 CREATE n:127.0.0.1:52721_solr c:testcollection s:shard1) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@bc406a[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2845238 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@48c116b5[testcollection_shard2_replica_n6] main]
   [junit4]   2> 2845240 INFO  (parallelCoreAdminExecutor-7018-thread-1-processing-n:127.0.0.1:52721_solr x:testcollection_shard1_replica_n1 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249334178897 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
   [junit4]   2> 2845240 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2845241 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2845242 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2845242 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1605221329279123456
   [junit4]   2> 2845247 INFO  (parallelCoreAdminExecutor-7019-thread-2-processing-n:127.0.0.1:59685_solr x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
   [junit4]   2> 2845249 INFO  (searcherExecutor-7028-thread-1-processing-x:testcollection_shard2_replica_n6 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249345115649 r:core_node8 CREATE n:127.0.0.1:59685_solr c:testcollection s:shard2) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] Registered new searcher Searcher@48c116b5[testcollection_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2845540 INFO  (qtp1514045924-19848) [n:127.0.0.1:59685_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2845541 INFO  (qtp1514045924-19848) [n:127.0.0.1:59685_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a2874e6-04e3-4691-b1dd-586c6490b5e6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2845719 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2845720 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2845720 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52721/solr/testcollection_shard2_replica_n4/
   [junit4]   2> 2845720 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n4 url=http://127.0.0.1:52721/solr START replicas=[http://127.0.0.1:59685/solr/testcollection_shard2_replica_n6/] nUpdates=100
   [junit4]   2> 2845721 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n4 url=http://127.0.0.1:52721/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2845723 INFO  (qtp1514045924-19852) [n:127.0.0.1:59685_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.S.Request [testcollection_shard2_replica_n6]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2845723 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2845723 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2845723 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2845727 INFO  (parallelCoreAdminExecutor-7018-thread-2-processing-n:127.0.0.1:52721_solr x:testcollection_shard2_replica_n4 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249342038157 CREATE) [n:127.0.0.1:52721_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52721/solr/testcollection_shard2_replica_n4/ shard2
   [junit4]   2> 2845735 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2845735 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2845735 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59685/solr/testcollection_shard1_replica_n2/
   [junit4]   2> 2845736 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:59685/solr START replicas=[http://127.0.0.1:52721/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 2845740 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:59685/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2845742 INFO  (qtp1795113159-19849) [n:127.0.0.1:52721_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2845743 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2845743 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2845743 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2845753 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e631110249338878288 CREATE) [n:127.0.0.1:59685_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59685/solr/testcollection_shard1_replica_n2/ shard1
   [junit4]   2> 2845855 INFO  (parallelCoreAdminExecutor-7019-thread-1-processing-n:127.0.0.1:59685_solr x:testcollection_shard1_replica_n2 0a2874e6-04e3-4691-b1dd-586c6490b5e6311102493388782

[...truncated too long message...]

workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null831816451
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null831816451
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null831816451

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 99ms :: artifacts dl 9ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 177 minutes 48 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 90 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/90/

6 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLowerTerms

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:7047561CB4A3FA00]:0)
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:331)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:299)
	at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
	at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.server.Server.doStart(Server.java:398)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369)
	at org.apache.solr.cloud.ForceLeaderTest.bringBackOldLeaderAndSendDoc(ForceLeaderTest.java:396)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLowerTerms(ForceLeaderTest.java:144)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:39833/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:39833/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:5D89D8FAF3ECB0D1]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:170)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39833/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more


FAILED:  org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode

Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=5},status={state=notfound,msg=Did not find [search_rate_trigger3/6df7c66b973658T1f7726pjk2wwhro1lfpym82s1/0] in any tasks queue}}

Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=5},status={state=notfound,msg=Did not find [search_rate_trigger3/6df7c66b973658T1f7726pjk2wwhro1lfpym82s1/0] in any tasks queue}}
	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:DCE1B8DD43CED509]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:670)
	at java.util.ArrayList.forEach(ArrayList.java:1257)
	at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:662)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost

Error Message:
did not finish processing all events in time: started=5, finished=4

Stack Trace:
java.lang.AssertionError: did not finish processing all events in time: started=5, finished=4
	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:4166B8A1F7EE3FF2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:525)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:378)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:599C6BFCBB49D56C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:309)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([FE73765F74045A74:7554A58E3502F1F0]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12759 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestLargeCluster_FE73765F74045A74-001/init-core-data-001
   [junit4]   2> 496005 WARN  (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 496006 INFO  (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 496007 INFO  (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 496007 INFO  (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 496110 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 496110 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 496111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 496111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 496118 DEBUG (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 496129 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 496145 INFO  (SUITE-TestLargeCluster-seed#[FE73765F74045A74]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 496152 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496152 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 496152 DEBUG (ScheduledTrigger-2005-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 27
   [junit4]   2> 496179 DEBUG (ScheduledTrigger-2005-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 86
   [junit4]   2> 496196 INFO  (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 496196 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 496196 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 496197 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 496198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 496199 INFO  (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 496199 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 496201 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 496207 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 496207 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 496207 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 496207 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 496207 DEBUG (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 496219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 496219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 496220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 496220 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 496232 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 496240 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496240 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 496240 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496283 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496303 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496324 DEBUG (ScheduledTrigger-2010-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496344 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496364 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496385 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496389 DEBUG (simCloudManagerPool-2009-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 496390 DEBUG (simCloudManagerPool-2009-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 496406 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496426 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 496430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496430 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 496447 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496452 DEBUG (simCloudManagerPool-2009-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 496453 DEBUG (simCloudManagerPool-2009-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 496467 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496480 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 496487 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 496492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 496492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 496666 DEBUG (simCloudManagerPool-2009-thread-5) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 496667 DEBUG (simCloudManagerPool-2009-thread-5) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 496679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 496679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger1 - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 496679 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496680 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 496680 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496685 DEBUG (simCloudManagerPool-2009-thread-6) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 496685 DEBUG (simCloudManagerPool-2009-thread-6) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 496700 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496700 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 496701 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 496701 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 496722 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496743 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496763 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496784 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496804 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496824 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496845 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496865 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496885 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496906 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496926 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496946 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496967 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 496987 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497007 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497028 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497048 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497181 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497202 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497222 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497242 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497263 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497283 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497303 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497324 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497344 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497364 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497384 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497405 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497425 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497445 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497466 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497486 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497506 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497527 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497547 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497567 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497588 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497681 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497702 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497722 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497742 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497763 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497783 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497803 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497869 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497889 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497897 DEBUG (simCloudManagerPool-2009-thread-83) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard2: {"core_node17":{
   [junit4]   2>     "core":"testBasic_shard2_replica_n17",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10029_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 497910 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497918 DEBUG (simCloudManagerPool-2009-thread-84) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node39":{
   [junit4]   2>     "core":"testBasic_shard3_replica_t39",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10060_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"TLOG",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard3",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 497918 DEBUG (simCloudManagerPool-2009-thread-85) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node50":{
   [junit4]   2>     "core":"testBasic_shard4_replica_n50",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard4",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 497918 DEBUG (simCloudManagerPool-2009-thread-86) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard5: {"core_node61":{
   [junit4]   2>     "core":"testBasic_shard5_replica_n61",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10082_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard5",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 497925 DEBUG (simCloudManagerPool-2009-thread-87) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard1: {"core_node1":{
   [junit4]   2>     "core":"testBasic_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10079_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 497930 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497950 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497971 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 497991 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498011 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498031 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498052 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498072 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498129 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498151 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498171 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498191 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498212 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498232 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498253 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498273 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498293 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498313 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498334 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498354 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498374 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 100
   [junit4]   2> 498391 INFO  (TEST-TestLargeCluster.testBasic-seed#[FE73765F74045A74]) [    ] o.a.s.c.a.s.TestLargeCluster Ready after 24025ms
   [junit4]   2> 498395 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 99
   [junit4]   2> 498395 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10026_solr
   [junit4]   2> 498415 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 99
   [junit4]   2> 498435 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 99
   [junit4]   2> 498456 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 98
   [junit4]   2> 498456 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10016_solr
   [junit4]   2> 498476 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 98
   [junit4]   2> 498496 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 98
   [junit4]   2> 498497 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10026_solr]
   [junit4]   2> 498500 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6df53833e0c66fT1f7726pjk2wwhro1lfpym7xhi",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950394198804079,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950394198804079],
   [junit4]   2>     "nodeNames":["127.0.0.1:10026_solr"]}}
   [junit4]   2> 498539 DEBUG (ScheduledTrigger-2010-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
   [junit4]   2> 498824 INFO  (simCloudManagerPool-2009-thread-127) [    ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 4890ms
   [junit4]   2> 498828 DEBUG (simCloudManagerPool-2009-thread-99) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 498840 DEBUG (simCloudManagerPool-2009-thread-101) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 498844 DEBUG (simCloudManagerPool-2009-thread-122) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node54":{
   [junit4]   2>     "core":"testBasic_shard4_replica_t54",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10087_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"TLOG",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard4",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 498882 DEBUG (simCloudManagerPool-2009-thread-130) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 498886 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6df53833e0c66fT1f7726pjk2wwhro1lfpym7xhi",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950394198804079,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950394198804079],
   [junit4]   2>     "_enqueue_time_":30950401464535479,
   [junit4]   2>     "nodeNames":["127.0.0.1:10026_solr"]}}
   [junit4]   2> 498901 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6df53833e0c66fT1f7726pjk2wwhro1lfpym7xhi",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950394198804079,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950394198804079],
   [junit4]   2>     "_enqueue_time_":30950401464535479,
   [junit4]   2>     "nodeNames":["127.0.0.1:10026_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
   [junit4]   2> 498911 DEBUG (simCloudManagerPool-2009-thread-131) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 498923 DEBUG (simCloudManagerPool-2009-thread-132) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 499274 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node19
   [junit4]   2> 499277 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30950438315 sessionWrapper.createTime 30950423879844829, this.sessionWrapper.createTime 30950423879844829 
   [junit4]   2> 499277 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"6df53833e0c66fT1f7726pjk2wwhro1lfpym7xhi",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950394198804079,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950394198804079],
   [junit4]   2>     "_enqueue_time_":30950401464535479,
   [junit4]   2>     "nodeNames":["127.0.0.1:10026_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@41cd0cb], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
   [junit4]   2> 499278 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node19
   [junit4]   2> 499379 DEBUG (simCloudManagerPool-2009-thread-134) [    ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node77":{
   [junit4]   2>     "core":"testBasic_shard2_replica_n76",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testBasic",
   [junit4]   2>     "node_name":"127.0.0.1:10072_solr",
   [junit4]   2>     "type":"NRT"}}
   [junit4]   2> 499609 DEBUG (simCloudManagerPool-2009-thread-142) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 499611 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 499612 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 36256 ms for event id=6df53833e0c66fT1f7726pjk2wwhro1lfpym7xhi
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10066_solr
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10022_solr
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10095_solr
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10055_solr
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10025_solr
   [junit4]   2> 499713 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10011_solr
   [junit4]   2> 499714 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10016_solr]
   [junit4]   2> 499714 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6df538e9e4008fT1f7726pjk2wwhro1lfpym7xhl",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950397252468879,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950397252468879],
   [junit4]   2>     "nodeNames":["127.0.0.1:10016_solr"]}}
   [junit4]   2> 499714 DEBUG (ScheduledTrigger-2010-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
   [junit4]   2> 499734 DEBUG (simCloudManagerPool-2009-thread-143) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 499734 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6df538e9e4008fT1f7726pjk2wwhro1lfpym7xhl",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950397252468879,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950397252468879],
   [junit4]   2>     "_enqueue_time_":30950460146581979,
   [junit4]   2>     "nodeNames":["127.0.0.1:10016_solr"]}}
   [junit4]   2> 499734 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6df538e9e4008fT1f7726pjk2wwhro1lfpym7xhl",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950397252468879,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950397252468879],
   [junit4]   2>     "_enqueue_time_":30950460146581979,
   [junit4]   2>     "nodeNames":["127.0.0.1:10016_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
   [junit4]   2> 499855 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node9
   [junit4]   2> 499857 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30950467294 sessionWrapper.createTime 30950461302797929, this.sessionWrapper.createTime 30950461302797929 
   [junit4]   2> 499857 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"6df538e9e4008fT1f7726pjk2wwhro1lfpym7xhl",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950397252468879,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[30950397252468879],
   [junit4]   2>     "_enqueue_time_":30950460146581979,
   [junit4]   2>     "nodeNames":["127.0.0.1:10016_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@3140badf], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
   [junit4]   2> 499857 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node9
   [junit4]   2> 499978 DEBUG (simCloudManagerPool-2009-thread-144) [    ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node79":{
   [junit4]   2>     "core":"testBasic_shard1_replica_t78",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testBasic",
   [junit4]   2>     "node_name":"127.0.0.1:10056_solr",
   [junit4]   2>     "type":"TLOG"}}
   [junit4]   2> 500235 DEBUG (simCloudManagerPool-2009-thread-152) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 500235 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 100ms
   [junit4]   2> 500235 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 25038 ms for event id=6df538e9e4008fT1f7726pjk2wwhro1lfpym7xhl
   [junit4]   2> 500335 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 92
   [junit4]   2> 500336 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10022_solr, 127.0.0.1:10055_solr, 127.0.0.1:10025_solr, 127.0.0.1:10066_solr, 127.0.0.1:10095_solr, 127.0.0.1:10011_solr]
   [junit4]   2> 500336 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6df5478d9dda7bT1f7726pjk2wwhro1lfpym7xho",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950460128877179,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[
   [junit4]   2>       30950460128877179,
   [junit4]   2>       30950460130752729,
   [junit4]   2>       30950460131666129,
   [junit4]   2>       30950460127525379,
   [junit4]   2>       30950460129830179,
   [junit4]   2>       30950460132575379],
   [junit4]   2>     "nodeNames":[
   [junit4]   2>       "127.0.0.1:10022_solr",
   [junit4]   2>       "127.0.0.1:10055_solr",
   [junit4]   2>       "127.0.0.1:10025_solr",
   [junit4]   2>       "127.0.0.1:10066_solr",
   [junit4]   2>       "127.0.0.1:10095_solr",
   [junit4]   2>       "127.0.0.1:10011_solr"]}}
   [junit4]   2> 500336 DEBUG (ScheduledTrigger-2010-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger1]
   [junit4]   2> 500337 DEBUG (simCloudManagerPool-2009-thread-153) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10091_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 500337 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6df5478d9dda7bT1f7726pjk2wwhro1lfpym7xho",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950460128877179,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[
   [junit4]   2>       30950460128877179,
   [junit4]   2>       30950460130752729,
   [junit4]   2>       30950460131666129,
   [junit4]   2>       30950460127525379,
   [junit4]   2>       30950460129830179,
   [junit4]   2>       30950460132575379],
   [junit4]   2>     "_enqueue_time_":30950491252585029,
   [junit4]   2>     "nodeNames":[
   [junit4]   2>       "127.0.0.1:10022_solr",
   [junit4]   2>       "127.0.0.1:10055_solr",
   [junit4]   2>       "127.0.0.1:10025_solr",
   [junit4]   2>       "127.0.0.1:10066_solr",
   [junit4]   2>       "127.0.0.1:10095_solr",
   [junit4]   2>       "127.0.0.1:10011_solr"]}}
   [junit4]   2> 500338 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6df5478d9dda7bT1f7726pjk2wwhro1lfpym7xho",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950460128877179,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[
   [junit4]   2>       30950460128877179,
   [junit4]   2>       30950460130752729,
   [junit4]   2>       30950460131666129,
   [junit4]   2>       30950460127525379,
   [junit4]   2>       30950460129830179,
   [junit4]   2>       30950460132575379],
   [junit4]   2>     "_enqueue_time_":30950491252585029,
   [junit4]   2>     "nodeNames":[
   [junit4]   2>       "127.0.0.1:10022_solr",
   [junit4]   2>       "127.0.0.1:10055_solr",
   [junit4]   2>       "127.0.0.1:10025_solr",
   [junit4]   2>       "127.0.0.1:10066_solr",
   [junit4]   2>       "127.0.0.1:10095_solr",
   [junit4]   2>       "127.0.0.1:10011_solr"]}} with context properties: {AFTER_ACTION=[start], BEFORE_ACTION=[start, compute]}
   [junit4]   2> 500693 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node23
   [junit4]   2> 500851 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node50
   [junit4]   2> 500914 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node60
   [junit4]   2> 500960 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10094_solr&inPlaceMove=true&replica=core_node34
   [junit4]   2> 501008 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10034_solr&inPlaceMove=true&replica=core_node70
   [junit4]   2> 501025 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10031_solr&inPlaceMove=true&replica=core_node72
   [junit4]   2> 501025 INFO  (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30950525711 sessionWrapper.createTime 30950491440165479, this.sessionWrapper.createTime 30950491440165479 
   [junit4]   2> 501025 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"6df5478d9dda7bT1f7726pjk2wwhro1lfpym7xho",
   [junit4]   2>   "source":"node_lost_trigger1",
   [junit4]   2>   "eventTime":30950460128877179,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[
   [junit4]   2>       30950460128877179,
   [junit4]   2>       30950460130752729,
   [junit4]   2>       30950460131666129,
   [junit4]   2>       30950460127525379,
   [junit4]   2>       30950460129830179,
   [junit4]   2>       30950460132575379],
   [junit4]   2>     "_enqueue_time_":30950491252585029,
   [junit4]   2>     "nodeNames":[
   [junit4]   2>       "127.0.0.1:10022_solr",
   [junit4]   2>       "127.0.0.1:10055_solr",
   [junit4]   2>       "127.0.0.1:10025_solr",
   [junit4]   2>       "127.0.0.1:10066_solr",
   [junit4]   2>       "127.0.0.1:10095_solr",
   [junit4]   2>       "127.0.0.1:10011_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@2f420b25, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6f248a7e, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6794e605, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@649be8e, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@39ed8641, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@72d19c78], AFTER_ACTION=[start, compute], BEFORE_ACTION=[start, compute, execute]}
   [junit4]   2> 501025 DEBUG (AutoscalingActionExecutor-2011-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node23
   [junit4]   2> 501137 DEBUG (simCloudManagerPool-2009-thread-154) [    ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node81"

[...truncated too long message...]

rkspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null960204395
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null960204395
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null960204395

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 123ms :: artifacts dl 212ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 176 minutes 7 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 89 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/89/

5 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.test

Error Message:
Error from server at http://127.0.0.1:37893: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37893: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
	at __randomizedtesting.SeedInfo.seed([FDA0F4BA9EEE4B5C:75F4CB60301226A4]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:416)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1006)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.api.collections.ShardSplitTest

Error Message:
52 threads leaked from SUITE scope at org.apache.solr.cloud.api.collections.ShardSplitTest:     1) Thread[id=7516, name=zkConnectionManagerCallback-1908-thread-1-EventThread, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)    2) Thread[id=7533, name=SolrRrdBackendFactory-3080-thread-2, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=7459, name=MetricsHistoryHandler-3081-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=7515, name=zkConnectionManagerCallback-1908-thread-1-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)    5) Thread[id=7470, name=qtp1190328540-7470, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$243/2091435795.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=7440, name=Scheduler-1476222663, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=7452, name=zkConnectionManagerCallback-1901-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=7436, name=qtp1973973368-7436, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=7432, name=qtp1973973368-7432, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=7445, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=7454, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=7502, name=zkConnectionManagerCallback-1924-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=7534, name=SolrRrdBackendFactory-3094-thread-2, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=7474, name=qtp1190328540-7474, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=7504, name=MetricsHistoryHandler-3095-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=7472, name=qtp1190328540-7472-acceptor-0@db77461-ServerConnector@4f9d9b34{HTTP/1.1,[http/1.1]}{127.0.0.1:41893}, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=7473, name=qtp1190328540-7473, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=7496, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   19) Thread[id=7460, name=Thread-1405, state=WAITING, group=TGRP-ShardSplitTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)   20) Thread[id=7500, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   21) Thread[id=7503, name=SolrRrdBackendFactory-3094-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   22) Thread[id=7476, name=qtp1190328540-7476, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   23) Thread[id=7451, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   24) Thread[id=7499, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   25) Thread[id=7523, name=OverseerCollectionConfigSetProcessor-72904060971188234-127.0.0.1:34339_-n_0000000001, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:317)         at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=7457, name=zkConnectionManagerCallback-1908-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=7458, name=SolrRrdBackendFactory-3080-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   28) Thread[id=7444, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   29) Thread[id=7505, name=Thread-1414, state=WAITING, group=TGRP-ShardSplitTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)   30) Thread[id=7446, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   31) Thread[id=7478, name=Scheduler-1120834440, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   32) Thread[id=7489, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   33) Thread[id=7437, name=qtp1973973368-7437, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   34) Thread[id=7475, name=qtp1190328540-7475, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   35) Thread[id=7434, name=qtp1973973368-7434-acceptor-0@277a1cc0-ServerConnector@59cc55ec{HTTP/1.1,[http/1.1]}{127.0.0.1:34339}, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   36) Thread[id=7491, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   37) Thread[id=7439, name=qtp1973973368-7439, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   38) Thread[id=7524, name=OverseerAutoScalingTriggerThread-72904060971188234-127.0.0.1:34339_-n_0000000001, state=WAITING, group=Overseer autoscaling triggers]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)         at java.lang.Thread.run(Thread.java:748)   39) Thread[id=7462, name=Scheduler-1144962756, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   40) Thread[id=7466, name=searcherExecutor-3083-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   41) Thread[id=7522, name=OverseerStateUpdate-72904060971188234-127.0.0.1:34339_-n_0000000001, state=TIMED_WAITING, group=Overseer state updater.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:417)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145)         at java.lang.Thread.run(Thread.java:748)   42) Thread[id=7433, name=qtp1973973368-7433, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   43) Thread[id=7469, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   44) Thread[id=7438, name=qtp1973973368-7438, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   45) Thread[id=7477, name=qtp1190328540-7477, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   46) Thread[id=7495, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   47) Thread[id=7435, name=qtp1973973368-7435, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.lang.Thread.run(Thread.java:748)   48) Thread[id=7497, name=zkConnectionManagerCallback-1917-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   49) Thread[id=7501, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   50) Thread[id=7450, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   51) Thread[id=7471, name=qtp1190328540-7471, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$243/2091435795.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   52) Thread[id=7490, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 52 threads leaked from SUITE scope at org.apache.solr.cloud.api.collections.ShardSplitTest: 
   1) Thread[id=7516, name=zkConnectionManagerCallback-1908-thread-1-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   2) Thread[id=7533, name=SolrRrdBackendFactory-3080-thread-2, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=7459, name=MetricsHistoryHandler-3081-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=7515, name=zkConnectionManagerCallback-1908-thread-1-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   5) Thread[id=7470, name=qtp1190328540-7470, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at org.eclipse.jetty.io.ManagedSelector$$Lambda$243/2091435795.run(Unknown Source)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=7440, name=Scheduler-1476222663, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=7452, name=zkConnectionManagerCallback-1901-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=7436, name=qtp1973973368-7436, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
   9) Thread[id=7432, name=qtp1973973368-7432, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=7445, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  11) Thread[id=7454, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  12) Thread[id=7502, name=zkConnectionManagerCallback-1924-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  13) Thread[id=7534, name=SolrRrdBackendFactory-3094-thread-2, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=7474, name=qtp1190328540-7474, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  15) Thread[id=7504, name=MetricsHistoryHandler-3095-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  16) Thread[id=7472, name=qtp1190328540-7472-acceptor-0@db77461-ServerConnector@4f9d9b34{HTTP/1.1,[http/1.1]}{127.0.0.1:41893}, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
  17) Thread[id=7473, name=qtp1190328540-7473, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  18) Thread[id=7496, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  19) Thread[id=7460, name=Thread-1405, state=WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
  20) Thread[id=7500, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
  21) Thread[id=7503, name=SolrRrdBackendFactory-3094-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  22) Thread[id=7476, name=qtp1190328540-7476, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  23) Thread[id=7451, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  24) Thread[id=7499, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  25) Thread[id=7523, name=OverseerCollectionConfigSetProcessor-72904060971188234-127.0.0.1:34339_-n_0000000001, state=TIMED_WAITING, group=Overseer collection creation process.]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
        at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:317)
        at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
        at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
        at java.lang.Thread.run(Thread.java:748)
  26) Thread[id=7457, name=zkConnectionManagerCallback-1908-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  27) Thread[id=7458, name=SolrRrdBackendFactory-3080-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  28) Thread[id=7444, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  29) Thread[id=7505, name=Thread-1414, state=WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)
  30) Thread[id=7446, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  31) Thread[id=7478, name=Scheduler-1120834440, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  32) Thread[id=7489, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  33) Thread[id=7437, name=qtp1973973368-7437, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  34) Thread[id=7475, name=qtp1190328540-7475, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  35) Thread[id=7434, name=qtp1973973368-7434-acceptor-0@277a1cc0-ServerConnector@59cc55ec{HTTP/1.1,[http/1.1]}{127.0.0.1:34339}, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
  36) Thread[id=7491, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  37) Thread[id=7439, name=qtp1973973368-7439, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
  38) Thread[id=7524, name=OverseerAutoScalingTriggerThread-72904060971188234-127.0.0.1:34339_-n_0000000001, state=WAITING, group=Overseer autoscaling triggers]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)
        at java.lang.Thread.run(Thread.java:748)
  39) Thread[id=7462, name=Scheduler-1144962756, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  40) Thread[id=7466, name=searcherExecutor-3083-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  41) Thread[id=7522, name=OverseerStateUpdate-72904060971188234-127.0.0.1:34339_-n_0000000001, state=TIMED_WAITING, group=Overseer state updater.]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
        at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341)
        at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:417)
        at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145)
        at java.lang.Thread.run(Thread.java:748)
  42) Thread[id=7433, name=qtp1973973368-7433, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  43) Thread[id=7469, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  44) Thread[id=7438, name=qtp1973973368-7438, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  45) Thread[id=7477, name=qtp1190328540-7477, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  46) Thread[id=7495, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
  47) Thread[id=7435, name=qtp1973973368-7435, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.lang.Thread.run(Thread.java:748)
  48) Thread[id=7497, name=zkConnectionManagerCallback-1917-thread-1, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  49) Thread[id=7501, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-EventThread, state=WAITING, group=TGRP-ShardSplitTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  50) Thread[id=7450, name=TEST-ShardSplitTest.test-seed#[FDA0F4BA9EEE4B5C]-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
  51) Thread[id=7471, name=qtp1190328540-7471, state=RUNNABLE, group=TGRP-ShardSplitTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at org.eclipse.jetty.io.ManagedSelector$$Lambda$243/2091435795.run(Unknown Source)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.lang.Thread.run(Thread.java:748)
  52) Thread[id=7490, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ShardSplitTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
	at __randomizedtesting.SeedInfo.seed([FDA0F4BA9EEE4B5C]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.api.collections.ShardSplitTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=7533, name=SolrRrdBackendFactory-3080-thread-2, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=7457, name=zkConnectionManagerCallback-1908-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=7458, name=SolrRrdBackendFactory-3080-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=7459, name=MetricsHistoryHandler-3081-thread-1, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=7515, name=zkConnectionManagerCallback-1908-thread-1-SendThread(127.0.0.1:46549), state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)    6) Thread[id=7505, name=Thread-1414, state=WAITING, group=TGRP-ShardSplitTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1832)    7) Thread[id=7470, name=qtp1190328540-7470, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:105)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:401)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$243/2091435795.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=7440, name=Scheduler-1476222663, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=7478, name=Scheduler-1120834440, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=7452, name=zkConnectionManagerCallback-1901-thread-1, state=WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=7436, name=qtp1973973368-7436, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:105)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:401)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=7432, name=qtp1973973368-7432, state=TIMED_WAITING, group=TGRP-ShardSplitTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=7439, name=qtp1973973368-7439, state=RUNNABLE, group=TGRP-ShardSplitTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYo

[...truncated too long message...]

ests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null763479301
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null763479301
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null763479301

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 78ms :: artifacts dl 1ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 177 minutes 40 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 88 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/88/

6 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes

Error Message:
unexpected shard state expected:<inactive> but was:<active>

Stack Trace:
java.lang.AssertionError: unexpected shard state expected:<inactive> but was:<active>
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:C59A8ECFC767B6E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.verifyShard(ShardSplitTest.java:380)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitMixedReplicaTypes(ShardSplitTest.java:372)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/169)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard1_1_0":{       "state":"active",       "replicas":{         "core_node14":{           "core":"testSplitIntegration_collection_shard1_1_0_replica1",           "SEARCHER.searcher.maxDoc":6,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":23480,           "node_name":"127.0.0.1:10129_solr",           "base_url":"http://127.0.0.1:10129/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.1867454051971436E-5,           "SEARCHER.searcher.numDocs":6},         "core_node13":{           "core":"testSplitIntegration_collection_shard1_1_0_replica0",           "SEARCHER.searcher.maxDoc":6,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":23480,           "node_name":"127.0.0.1:10130_solr",           "base_url":"http://127.0.0.1:10130/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.1867454051971436E-5,           "SEARCHER.searcher.numDocs":6}}},     "shard2":{       "replicas":{         "core_node3":{           "core":"testSplitIntegration_collection_shard2_replica_n3",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10129_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "core":"testSplitIntegration_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10130_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"0-7fffffff",       "state":"active"},     "shard1_1_1":{       "state":"active",       "replicas":{"core_node15":{           "core":"testSplitIntegration_collection_shard1_1_1_replica0",           "SEARCHER.searcher.maxDoc":6,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":13240,           "node_name":"127.0.0.1:10130_solr",           "base_url":"http://127.0.0.1:10130/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.2330710887908936E-5,           "SEARCHER.searcher.numDocs":6}}},     "shard1":{       "stateTimestamp":"1530591564065884800",       "replicas":{         "core_node1":{           "core":"testSplitIntegration_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":25,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":22740,           "node_name":"127.0.0.1:10129_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.117827534675598E-5,           "SEARCHER.searcher.numDocs":25},         "core_node2":{           "core":"testSplitIntegration_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":25,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":22740,           "node_name":"127.0.0.1:10130_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.117827534675598E-5,           "SEARCHER.searcher.numDocs":25}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1530591564066455800",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":12,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":16240,           "node_name":"127.0.0.1:10129_solr",           "base_url":"http://127.0.0.1:10129/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.5124678611755371E-5,           "SEARCHER.searcher.numDocs":12},         "core_node9":{           "core":"testSplitIntegration_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":12,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":16240,           "node_name":"127.0.0.1:10130_solr",           "base_url":"http://127.0.0.1:10130/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.5124678611755371E-5,           "SEARCHER.searcher.numDocs":12}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1530591564066328550",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":13,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":26980,           "node_name":"127.0.0.1:10130_solr",           "base_url":"http://127.0.0.1:10130/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.512708306312561E-5,           "SEARCHER.searcher.numDocs":13},         "core_node8":{           "core":"testSplitIntegration_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":13,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":26980,           "node_name":"127.0.0.1:10129_solr",           "base_url":"http://127.0.0.1:10129/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.512708306312561E-5,           "SEARCHER.searcher.numDocs":13}}}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/169)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard1_1_0":{
      "state":"active",
      "replicas":{
        "core_node14":{
          "core":"testSplitIntegration_collection_shard1_1_0_replica1",
          "SEARCHER.searcher.maxDoc":6,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":23480,
          "node_name":"127.0.0.1:10129_solr",
          "base_url":"http://127.0.0.1:10129/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.1867454051971436E-5,
          "SEARCHER.searcher.numDocs":6},
        "core_node13":{
          "core":"testSplitIntegration_collection_shard1_1_0_replica0",
          "SEARCHER.searcher.maxDoc":6,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":23480,
          "node_name":"127.0.0.1:10130_solr",
          "base_url":"http://127.0.0.1:10130/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.1867454051971436E-5,
          "SEARCHER.searcher.numDocs":6}}},
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testSplitIntegration_collection_shard2_replica_n3",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10129_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testSplitIntegration_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10130_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1_1_1":{
      "state":"active",
      "replicas":{"core_node15":{
          "core":"testSplitIntegration_collection_shard1_1_1_replica0",
          "SEARCHER.searcher.maxDoc":6,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":13240,
          "node_name":"127.0.0.1:10130_solr",
          "base_url":"http://127.0.0.1:10130/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.2330710887908936E-5,
          "SEARCHER.searcher.numDocs":6}}},
    "shard1":{
      "stateTimestamp":"1530591564065884800",
      "replicas":{
        "core_node1":{
          "core":"testSplitIntegration_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":25,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":22740,
          "node_name":"127.0.0.1:10129_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.117827534675598E-5,
          "SEARCHER.searcher.numDocs":25},
        "core_node2":{
          "core":"testSplitIntegration_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":25,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":22740,
          "node_name":"127.0.0.1:10130_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.117827534675598E-5,
          "SEARCHER.searcher.numDocs":25}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1530591564066455800",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":12,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":16240,
          "node_name":"127.0.0.1:10129_solr",
          "base_url":"http://127.0.0.1:10129/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.5124678611755371E-5,
          "SEARCHER.searcher.numDocs":12},
        "core_node9":{
          "core":"testSplitIntegration_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":12,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":16240,
          "node_name":"127.0.0.1:10130_solr",
          "base_url":"http://127.0.0.1:10130/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.5124678611755371E-5,
          "SEARCHER.searcher.numDocs":12}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1530591564066328550",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":13,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":26980,
          "node_name":"127.0.0.1:10130_solr",
          "base_url":"http://127.0.0.1:10130/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.512708306312561E-5,
          "SEARCHER.searcher.numDocs":13},
        "core_node8":{
          "core":"testSplitIntegration_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":13,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":26980,
          "node_name":"127.0.0.1:10129_solr",
          "base_url":"http://127.0.0.1:10129/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.512708306312561E-5,
          "SEARCHER.searcher.numDocs":13}}}}}
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:8D14450C2F5267E5]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:312)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:3CA775333A6D4FB6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState(TestTriggerIntegration.java:383)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:E9D2E2C5CF6B0854]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Captured an uncaught exception in thread: Thread[id=13211, name=cdcr-replicator-4674-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13211, name=cdcr-replicator-4674-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError: 1604822256668639232 != 1604822256237674496
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B]:0)
	at org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
	at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
	at org.apache.solr.handler.CdcrReplicatorScheduler.lambda$null$0(CdcrReplicatorScheduler.java:81)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:3FBD2F9D41AB059F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12336 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_B49AFC4C00ADAE1B-001/init-core-data-001
   [junit4]   2> 4013 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 4090 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.e.j.u.log Logging initialized @4138ms to org.eclipse.jetty.util.log.Slf4jLog
   [junit4]   2> 4120 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 4533 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 4621 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 4799 DEBUG (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 4856 INFO  (SUITE-TestTriggerIntegration-seed#[B49AFC4C00ADAE1B]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 4901 INFO  (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
   [junit4]   2> 4905 INFO  (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 4908 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 4908 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 4909 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 4909 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 4909 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 4909 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 4910 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 4923 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 5070 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 5070 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 5095 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 5100 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5101 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 5101 DEBUG (ScheduledTrigger-8-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5123 DEBUG (ScheduledTrigger-8-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5143 DEBUG (simCloudManagerPool-7-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5150 DEBUG (simCloudManagerPool-7-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 5151 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5152 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 5153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5153 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 5162 DEBUG (simCloudManagerPool-7-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5164 DEBUG (simCloudManagerPool-7-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 5166 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 5166 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 5166 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 5166 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 5167 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 5167 DEBUG (ScheduledTrigger-8-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 5168 DEBUG (ScheduledTrigger-8-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 5171 DEBUG (ScheduledTrigger-8-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5189 DEBUG (ScheduledTrigger-8-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 5189 DEBUG (ScheduledTrigger-8-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 5189 DEBUG (ScheduledTrigger-8-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 12812948231221762
   [junit4]   2> 5191 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 5210 DEBUG (ScheduledTrigger-8-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 5210 DEBUG (ScheduledTrigger-8-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 5211 DEBUG (simCloudManagerPool-7-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5212 DEBUG (ScheduledTrigger-8-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 5212 DEBUG (simCloudManagerPool-7-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 5213 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 5214 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 5214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 5214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 5214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10002_solr
   [junit4]   2> 5214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> 5214 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 5215 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
   [junit4]   2> 5215 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 5215 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [12812949486472662], now=12812949527657912
   [junit4]   2> 5217 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"2d854f46aa8dd6T4nkqq99srzs294hz7t4p9ei9v",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":12812949486472662,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[12812949486472662],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 5218 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
   [junit4]   2> 5316 DEBUG (simCloudManagerPool-7-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 5322 INFO  (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
   [junit4]   2> 5327 DEBUG (ScheduledTrigger-8-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> 5328 DEBUG (AutoscalingActionExecutor-9-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"2d854f46aa8dd6T4nkqq99srzs294hz7t4p9ei9v",
   [junit4]   2>   "source":"node_added_restore_trigger",
   [junit4]   2>   "eventTime":12812949486472662,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[12812949486472662],
   [junit4]   2>     "_enqueue_time_":12812949685332612,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 5349 DEBUG (simCloudManagerPool-7-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 5353 DEBUG (AutoscalingActionExecutor-9-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 5353 DEBUG (AutoscalingActionExecutor-9-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
   [junit4]   2> 5353 DEBUG (AutoscalingActionExecutor-9-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1291 ms for event id=2d854f46aa8dd6T4nkqq99srzs294hz7t4p9ei9v
   [junit4]   2> 5383 INFO  (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		3
   [junit4]   2> ## Empty nodes:	2
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[.system]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	0
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             2
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_restore_trigger
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> ##		- SUCCEEDED       1
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeAddedTriggerRestoreState -Dtests.seed=B49AFC4C00ADAE1B -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ga-IE -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.52s J2 | TestTriggerIntegration.testNodeAddedTriggerRestoreState <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:3CA775333A6D4FB6]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTriggerRestoreState(TestTriggerIntegration.java:383)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 5394 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
   [junit4]   2> 5394 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 5396 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 5398 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 5399 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5399 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 5400 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10002_solr
   [junit4]   2> 5404 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 5406 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 5413 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 5413 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 5414 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 5414 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 5414 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 5414 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 5427 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 5428 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 5429 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
   [junit4]   2> 5429 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 5439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 5441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 5442 DEBUG (ScheduledTrigger-11-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5462 DEBUG (ScheduledTrigger-11-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5483 DEBUG (ScheduledTrigger-11-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5504 DEBUG (ScheduledTrigger-11-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5528 DEBUG (ScheduledTrigger-11-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5529 DEBUG (simCloudManagerPool-10-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5530 DEBUG (simCloudManagerPool-10-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 5531 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 5531 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5531 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 5549 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5570 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5595 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5603 DEBUG (simCloudManagerPool-10-thread-6) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5606 DEBUG (simCloudManagerPool-10-thread-6) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 5607 DEBUG (simCloudManagerPool-10-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"collection1"}}
   [junit4]   2> 5613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 5613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 5620 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5628 DEBUG (simCloudManagerPool-10-thread-7) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 5629 DEBUG (simCloudManagerPool-10-thread-7) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 5630 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 5630 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 5630 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 5640 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5661 DEBUG (ScheduledTrigger-11-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5681 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5701 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5722 DEBUG (ScheduledTrigger-11-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 5738 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"2d855458431450T4nkqq99srzs294hz7t4p9ei9y",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":12812971256517712,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 5738 DEBUG (ScheduledTrigger-11-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
   [junit4]   2> 5768 DEBUG (simCloudManagerPool-10-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 5811 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"2d855458431450T4nkqq99srzs294hz7t4p9ei9y",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":12812971256517712,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":12812975710560062,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 5811 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"2d855458431450T4nkqq99srzs294hz7t4p9ei9y",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":12812971256517712,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":12812975710560062,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute]}
   [junit4]   2> 5832 INFO  (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 5833 INFO  (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 5834 INFO  (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 5834 INFO  (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 12812980491 sessionWrapper.createTime 12812979608309812, this.sessionWrapper.createTime 12812979608309812 
   [junit4]   2> 5835 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"2d855458431450T4nkqq99srzs294hz7t4p9ei9y",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":12812971256517712,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "__start__":3,
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":12812975710560062,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@6b56e833, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@7a27b140, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@2a1c5894], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
   [junit4]   2> 5836 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 5861 INFO  (simCloudManagerPool-10-thread-11) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica
   [junit4]   2> 5921 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
   [junit4]   2> 5934 INFO  (simCloudManagerPool-10-thread-15) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica
   [junit4]   2> 5983 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
   [junit4]   2> 5992 INFO  (simCloudManagerPool-10-thread-19) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
   [junit4]   2> 6032 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
   [junit4]   2> 6032 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[collection1, .system]
   [junit4]   2> ## Max replicas per node:	3
   [junit4]   2> ## Min replicas per node:	3
   [junit4]   2> ## Total replicas:		6
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ## * collection1		5
   [junit4]   2> ##		- active           5
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- ADDREPLICA         3
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- DELETESTATUS       2
   [junit4]   2> ##		- REQUESTSTATUS      2
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: search_rate_trigger
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=B49AFC4C00ADAE1B -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ga-IE -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.64s J2 | TestTriggerIntegration.testSearchRate <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B49AFC4C00ADAE1B:E9D2E2C5CF6B0854]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1204)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 6063 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
   [junit4]   2> 6063 DEBUG (simCloudManagerPool-10-thread-23) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 6064 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 6064 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 6065 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 6065 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 6065 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 6065 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 6065 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 6065 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 6075 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 6075 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 6075 WARN  (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:756) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:321) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	... 10 more
   [junit4]   2> 6077 DEBUG (AutoscalingActionExecutor-12-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 14628 ms for event id=2d855458431450T4nkqq99srzs294hz7t4p9ei9y
   [junit4]   2> 6077 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 6077 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 6077 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[B49AFC4C00ADAE1B]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 6091 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 6092 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 6093 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
   [junit4]   2> 6093 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 6107 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 6107 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 6108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 6108 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6129 DEBUG (ScheduledTrigger-14-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6152 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6172 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6193 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6194 DEBUG (simCloudManagerPool-13-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 6194 DEBUG (simCloudManagerPool-13-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 6212 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 6212 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 6212 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 6213 DEBUG (ScheduledTrigger-14-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6225 DEBUG (simCloudManagerPool-13-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 6225 DEBUG (simCloudManagerPool-13-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 6226 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 6226 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 6226 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 6226 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 6228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 6228 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 6228 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 6233 DEBUG (ScheduledTrigger-14-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 6236 DEBUG (simCloudManagerPool-13-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 6237 DEBUG (simCloudManagerPool-13-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 6239 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 6239 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 6248 DEBUG (ScheduledTrigger-14-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 6249 DEBUG (ScheduledTrigger-14-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 6249 DEBUG (ScheduledTrigger-14-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 12813001212162762
   [junit4]   2> 6254 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 6271 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 6271 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 6271 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [12813001212162762], now=12813002320013812
   [junit4]   2> 6271 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"2d855b51c1f2caT4nkqq99srzs294hz7t4p9eia1",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":12813001212162762,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[12813001212162762],
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 6272 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 6310 DEBUG (simCloudManagerPool-13-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 6310 DEBUG (ScheduledTrigger-14-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10003_solr
   [junit4]   2> 6311 DEBUG (AutoscalingActionExecutor-15-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"2d855b51c1f2caT4nkqq99srzs294hz7t4p9eia1",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":12813001212162762,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[12813001212162762],
   [junit4]   2>     "_enqueue_time_":12813002383986962,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 6314 DEBUG (simCloudManagerPool-13-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4

[...truncated too long message...]

sts-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null277163377
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null277163377
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null277163377

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 71ms :: artifacts dl 18ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 216 minutes 20 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 87 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/87/

3 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at https://127.0.0.1:37768/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:37768/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([3D1B3B37BA65E6C9:C3951B828BB8B9]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
	at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37768/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more


FAILED:  org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest

Error Message:
Tlog size exceeds the max size bound. Tlog path: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001/tlog/tlog.0000000000000000002, tlog size: 1265

Stack Trace:
java.lang.AssertionError: Tlog size exceeds the max size bound. Tlog path: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001/tlog/tlog.0000000000000000002, tlog size: 1265
	at __randomizedtesting.SeedInfo.seed([3D1B3B37BA65E6C9:2D55DEC8C1CBDF38]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.update.MaxSizeAutoCommitTest.getTlogFileSizes(MaxSizeAutoCommitTest.java:383)
	at org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest(MaxSizeAutoCommitTest.java:203)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([3D1B3B37BA65E6C9:B63CE8E6FB634D4D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13746 lines...]
   [junit4] Suite: org.apache.solr.update.MaxSizeAutoCommitTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001
   [junit4]   2> 2095375 WARN  (SUITE-MaxSizeAutoCommitTest-seed#[3D1B3B37BA65E6C9]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
   [junit4]   2> 2095375 INFO  (SUITE-MaxSizeAutoCommitTest-seed#[3D1B3B37BA65E6C9]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2095377 INFO  (SUITE-MaxSizeAutoCommitTest-seed#[3D1B3B37BA65E6C9]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2095377 INFO  (SUITE-MaxSizeAutoCommitTest-seed#[3D1B3B37BA65E6C9]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2095382 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting simpleTest
   [junit4]   2> 2095383 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2095384 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2095973 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2096008 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2096261 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2096930 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2096931 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2096990 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2097008 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2097008 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2097011 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2097081 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2097095 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2097214 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2097223 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 2097223 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2097223 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2097223 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001/]
   [junit4]   2> 2097225 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2097227 WARN  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2097282 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2097282 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2097283 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2097283 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2097285 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=66.5576171875, floorSegmentMB=1.8759765625, forceMergeDeletesPctAllowed=8.949208482230127, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8701368330596397, reclaimDeletesWeight=3.661011280764526
   [junit4]   2> 2097285 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a9fc2b8[collection1] main]
   [junit4]   2> 2097286 WARN  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 2097286 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2097288 INFO  (searcherExecutor-5654-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7a9fc2b8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2097288 INFO  (coreLoadExecutor-5653-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604632524770770944
   [junit4]   2> 2097291 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2097300 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0 (1604632524776013824), 1 (1604632524782305280), 2 (1604632524783353856)]} 0 8
   [junit4]   2> 2097606 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[3 (1604632525103169536), 4 (1604632525104218112), 5 (1604632525104218113)]} 0 1
   [junit4]   2> 2097809 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[6 (1604632525317079040), 7 (1604632525317079041), 8 (1604632525317079042)]} 0 0
   [junit4]   2> 2098011 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[9 (1604632525528891392), 10 (1604632525528891393), 11 (1604632525528891394)]} 0 0
   [junit4]   2> 2098213 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[12 (1604632525739655168), 13 (1604632525739655169), 14 (1604632525739655170)]} 0 0
   [junit4]   2> 2098418 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[15 (1604632525954613248), 16 (1604632525954613249), 17 (1604632525954613250)]} 0 0
   [junit4]   2> 2098619 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[18 (1604632526166425600), 19 (1604632526166425601), 20 (1604632526166425602)]} 0 0
   [junit4]   2> 2098820 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[21 (1604632526377189376), 22 (1604632526377189377), 23 (1604632526377189378)]} 0 0
   [junit4]   2> 2099029 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[24 (1604632526594244608), 25 (1604632526594244609), 26 (1604632526595293184)]} 0 1
   [junit4]   2> 2099029 INFO  (commitScheduler-5657-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2099029 INFO  (commitScheduler-5657-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@51590f1f commitCommandVersion:0
   [junit4]   2> 2099046 INFO  (commitScheduler-5657-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ef3fccb[collection1] realtime]
   [junit4]   2> 2099047 INFO  (commitScheduler-5657-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2099081 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[27 (1604632526648770560), 28 (1604632526649819136), 29 (1604632526650867712)]} 0 1
   [junit4]   2> 2099282 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending simpleTest
   [junit4]   2> 2099282 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2099282 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1849179677
   [junit4]   2> 2099282 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 2099282 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20f3ede1: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@47913eae
   [junit4]   2> 2099289 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 2099289 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@10cc8bac: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4fe896bb
   [junit4]   2> 2099293 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2099293 INFO  (TEST-MaxSizeAutoCommitTest.simpleTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7084190b: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@369031fa
   [junit4]   2> 2099300 INFO  (coreCloseExecutor-5659-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4fe36aa8
   [junit4]   2> 2099300 INFO  (coreCloseExecutor-5659-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=4fe36aa8
   [junit4]   2> 2099300 INFO  (coreCloseExecutor-5659-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@314bea91: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@70a9894f
   [junit4]   2> 2099310 INFO  (coreCloseExecutor-5659-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 2099310 INFO  (coreCloseExecutor-5659-thread-1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@51590f1f commitCommandVersion:0
   [junit4]   2> 2099323 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting endToEndTest {seed=[3D1B3B37BA65E6C9:274D54015AD7C1CB]}
   [junit4]   2> 2099323 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2099324 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2099351 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2099366 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2099498 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2099747 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2099747 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2099770 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2099797 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2099797 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2099800 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2099833 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2099847 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2100044 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2100052 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 2100053 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2100053 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2100053 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001/]
   [junit4]   2> 2100064 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2100105 WARN  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2100163 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2100164 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2100165 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2100165 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2100166 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=66.5576171875, floorSegmentMB=1.8759765625, forceMergeDeletesPctAllowed=8.949208482230127, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8701368330596397, reclaimDeletesWeight=3.661011280764526
   [junit4]   2> 2100167 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@61c36845[collection1] main]
   [junit4]   2> 2100167 WARN  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 2100168 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2100168 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604632527790669824
   [junit4]   2> 2100171 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2100175 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0 (1604632527794864128), 1 (1604632527795912704), 2 (1604632527795912705), 3 (1604632527795912706), 4 (1604632527795912707), 5 (1604632527795912708), 6 (1604632527796961280), 7 (1604632527796961281), 8 (1604632527796961282), 9 (1604632527796961283), ... (20 adds)]} 0 3
   [junit4]   2> 2100176 INFO  (searcherExecutor-5670-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@61c36845[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2100376 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=204
   [junit4]   2> 2100381 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[20 (1604632528011919360), 21 (1604632528011919361), 22 (1604632528011919362), 23 (1604632528011919363), 24 (1604632528011919364), 25 (1604632528011919365), 26 (1604632528011919366), 27 (1604632528012967936), 28 (1604632528012967937), 29 (1604632528012967938), ... (20 adds)]} 0 1
   [junit4]   2> 2100584 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=412
   [junit4]   2> 2100586 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[40 (1604632528226877440), 41 (1604632528226877441), 42 (1604632528226877442), 43 (1604632528226877443), 44 (1604632528227926016), 45 (1604632528227926017), 46 (1604632528227926018), 47 (1604632528227926019), 48 (1604632528227926020), 49 (1604632528227926021), ... (20 adds)]} 0 2
   [junit4]   2> 2100787 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=615
   [junit4]   2> 2100790 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[60 (1604632528440786944), 61 (1604632528440786945), 62 (1604632528440786946), 63 (1604632528440786947), 64 (1604632528440786948), 65 (1604632528440786949), 66 (1604632528441835520), 67 (1604632528441835521), 68 (1604632528441835522), 69 (1604632528441835523), ... (20 adds)]} 0 1
   [junit4]   2> 2100995 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=823
   [junit4]   2> 2100998 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[80 (1604632528658890752), 81 (1604632528658890753), 82 (1604632528658890754), 83 (1604632528658890755), 84 (1604632528658890756), 85 (1604632528658890757), 86 (1604632528658890758), 87 (1604632528658890759), 88 (1604632528658890760), 89 (1604632528659939328), ... (20 adds)]} 0 2
   [junit4]   2> 2101198 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1027
   [junit4]   2> 2101207 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[100 (1604632528878043136), 101 (1604632528878043137), 102 (1604632528878043138), 103 (1604632528879091712), 104 (1604632528879091713), 105 (1604632528879091714), 106 (1604632528879091715), 107 (1604632528879091716), 108 (1604632528879091717), 109 (1604632528879091718), ... (20 adds)]} 0 8
   [junit4]   2> 2101408 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1236
   [junit4]   2> 2101410 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[120 (1604632529090904064), 121 (1604632529090904065), 122 (1604632529091952640), 123 (1604632529091952641), 124 (1604632529091952642), 125 (1604632529091952643), 126 (1604632529091952644), 127 (1604632529091952645), 128 (1604632529091952646), 129 (1604632529091952647), ... (20 adds)]} 0 2
   [junit4]   2> 2101411 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2101411 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2101422 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@524fa77a[collection1] realtime]
   [junit4]   2> 2101422 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2101461 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1289
   [junit4]   2> 2101463 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[140 (1604632529146478592), 141 (1604632529147527168), 142 (1604632529147527169), 143 (1604632529147527170), 144 (1604632529147527171), 145 (1604632529147527172), 146 (1604632529147527173), 147 (1604632529147527174), 148 (1604632529147527175), 149 (1604632529147527176), ... (20 adds)]} 0 2
   [junit4]   2> 2101664 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1492
   [junit4]   2> 2101666 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[160 (1604632529359339520), 161 (1604632529360388096), 162 (1604632529360388097), 163 (1604632529360388098), 164 (1604632529360388099), 165 (1604632529360388100), 166 (1604632529360388101), 167 (1604632529360388102), 168 (1604632529360388103), 169 (1604632529360388104), ... (20 adds)]} 0 1
   [junit4]   2> 2101867 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1695
   [junit4]   2> 2101869 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[180 (1604632529573249024), 181 (1604632529573249025), 182 (1604632529573249026), 183 (1604632529573249027), 184 (1604632529573249028), 185 (1604632529573249029), 186 (1604632529573249030), 187 (1604632529573249031), 188 (1604632529573249032), 189 (1604632529573249033), ... (20 adds)]} 0 1
   [junit4]   2> 2102070 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1898
   [junit4]   2> 2102072 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[200 (1604632529786109952), 201 (1604632529786109953), 202 (1604632529786109954), 203 (1604632529786109955), 204 (1604632529786109956), 205 (1604632529786109957), 206 (1604632529786109958), 207 (1604632529786109959), 208 (1604632529786109960), 209 (1604632529786109961), ... (20 adds)]} 0 1
   [junit4]   2> 2102274 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2102
   [junit4]   2> 2102276 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[220 (1604632529998970880), 221 (1604632529998970881), 222 (1604632529998970882), 223 (1604632530000019456), 224 (1604632530000019457), 225 (1604632530000019458), 226 (1604632530000019459), 227 (1604632530000019460), 228 (1604632530000019461), 229 (1604632530000019462), ... (20 adds)]} 0 1
   [junit4]   2> 2102477 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2305
   [junit4]   2> 2102479 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[240 (1604632530211831808), 241 (1604632530212880384), 242 (1604632530212880385), 243 (1604632530212880386), 244 (1604632530212880387), 245 (1604632530212880388), 246 (1604632530212880389), 247 (1604632530212880390), 248 (1604632530212880391), 249 (1604632530212880392), ... (20 adds)]} 0 1
   [junit4]   2> 2102680 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2508
   [junit4]   2> 2102685 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2102685 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2102690 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[260 (1604632530424692736), 261 (1604632530425741312), 262 (1604632530425741313), 263 (1604632530425741314), 264 (1604632530425741315), 265 (1604632530427838464), 266 (1604632530427838465), 267 (1604632530427838466), 268 (1604632530427838467), 269 (1604632530427838468), ... (20 adds)]} 0 10
   [junit4]   2> 2102693 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@d869778[collection1] realtime]
   [junit4]   2> 2102693 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2102891 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2720
   [junit4]   2> 2102894 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[280 (1604632530646990848), 281 (1604632530646990849), 282 (1604632530648039424), 283 (1604632530648039425), 284 (1604632530648039426), 285 (1604632530648039427), 286 (1604632530648039428), 287 (1604632530648039429), 288 (1604632530648039430), 289 (1604632530648039431), ... (20 adds)]} 0 2
   [junit4]   2> 2103095 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2923
   [junit4]   2> 2103097 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[300 (1604632530859851776), 301 (1604632530860900352), 302 (1604632530860900353), 303 (1604632530860900354), 304 (1604632530860900355), 305 (1604632530860900356), 306 (1604632530860900357), 307 (1604632530860900358), 308 (1604632530860900359), 309 (1604632530860900360), ... (20 adds)]} 0 1
   [junit4]   2> 2103298 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=3126
   [junit4]   2> 2103300 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[320 (1604632531072712704), 321 (1604632531073761280), 322 (1604632531073761281), 323 (1604632531073761282), 324 (1604632531073761283), 325 (1604632531073761284), 326 (1604632531073761285), 327 (1604632531073761286), 328 (1604632531073761287), 329 (1604632531073761288), ... (20 adds)]} 0 1
   [junit4]   2> 2103501 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=3329
   [junit4]   2> 2103505 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[340 (1604632531288719360), 341 (1604632531288719361), 342 (1604632531288719362), 343 (1604632531288719363), 344 (1604632531288719364), 345 (1604632531288719365), 346 (1604632531288719366), 347 (1604632531288719367), 348 (1604632531288719368), 349 (1604632531288719369), ... (20 adds)]} 0 2
   [junit4]   2> 2103711 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=3539
   [junit4]   2> 2103713 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[360 (1604632531505774592), 361 (1604632531505774593), 362 (1604632531505774594), 363 (1604632531506823168), 364 (1604632531506823169), 365 (1604632531506823170), 366 (1604632531506823171), 367 (1604632531506823172), 368 (1604632531506823173), 369 (1604632531506823174), ... (20 adds)]} 0 1
   [junit4]   2> 2103914 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=3742
   [junit4]   2> 2103916 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[380 (1604632531718635520), 381 (1604632531719684096), 382 (1604632531719684097), 383 (1604632531719684098), 384 (1604632531719684099), 385 (1604632531719684100), 386 (1604632531719684101), 387 (1604632531719684102), 388 (1604632531719684103), 389 (1604632531719684104), ... (20 adds)]} 0 2
   [junit4]   2> 2104117 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=3945
   [junit4]   2> 2104128 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[400 (1604632531931496448), 401 (1604632531932545024), 402 (1604632531932545025), 403 (1604632531932545026), 404 (1604632531932545027), 405 (1604632531932545028), 406 (1604632531932545029), 407 (1604632531932545030), 408 (1604632531941982208), 409 (1604632531941982209), ... (20 adds)]} 0 10
   [junit4]   2> 2104128 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2104128 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2104132 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1beb239c[collection1] realtime]
   [junit4]   2> 2104132 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2104178 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=4006
   [junit4]   2> 2104182 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[420 (1604632531996508160), 421 (1604632531997556736), 422 (1604632531997556737), 423 (1604632531997556738), 424 (1604632531998605312), 425 (1604632531998605313), 426 (1604632531998605314), 427 (1604632531998605315), 428 (1604632531998605316), 429 (1604632531998605317), ... (20 adds)]} 0 3
   [junit4]   2> 2104383 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=4211
   [junit4]   2> 2104385 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[440 (1604632532210417664), 441 (1604632532210417665), 442 (1604632532210417666), 443 (1604632532210417667), 444 (1604632532211466240), 445 (1604632532211466241), 446 (1604632532211466242), 447 (1604632532211466243), 448 (1604632532211466244), 449 (1604632532211466245), ... (20 adds)]} 0 1
   [junit4]   2> 2104586 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=4414
   [junit4]   2> 2104587 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[460 (1604632532423278592), 461 (1604632532423278593), 462 (1604632532423278594), 463 (1604632532423278595), 464 (1604632532423278596), 465 (1604632532423278597), 466 (1604632532423278598), 467 (1604632532424327168), 468 (1604632532424327169), 469 (1604632532424327170), ... (20 adds)]} 0 1
   [junit4]   2> 2104788 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=4616
   [junit4]   2> 2104792 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[480 (1604632532638236672), 481 (1604632532638236673), 482 (1604632532638236674), 483 (1604632532638236675), 484 (1604632532638236676), 485 (1604632532638236677), 486 (1604632532638236678), 487 (1604632532638236679), 488 (1604632532638236680), 489 (1604632532639285248), ... (20 adds)]} 0 1
   [junit4]   2> 2104993 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=4821
   [junit4]   2> 2104995 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1604632532851097600), 501 (1604632532851097601), 502 (1604632532851097602), 503 (1604632532851097603), 504 (1604632532851097604), 505 (1604632532851097605), 506 (1604632532851097606), 507 (1604632532851097607), 508 (1604632532851097608), 509 (1604632532851097609), ... (20 adds)]} 0 1
   [junit4]   2> 2105196 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5024
   [junit4]   2> 2105198 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[520 (1604632533062909952), 521 (1604632533063958528), 522 (1604632533063958529), 523 (1604632533063958530), 524 (1604632533063958531), 525 (1604632533063958532), 526 (1604632533063958533), 527 (1604632533063958534), 528 (1604632533063958535), 529 (1604632533063958536), ... (20 adds)]} 0 1
   [junit4]   2> 2105399 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5227
   [junit4]   2> 2105401 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[540 (1604632533275770880), 541 (1604632533276819456), 542 (1604632533276819457), 543 (1604632533276819458), 544 (1604632533276819459), 545 (1604632533276819460), 546 (1604632533276819461), 547 (1604632533276819462), 548 (1604632533276819463), 549 (1604632533276819464), ... (20 adds)]} 0 1
   [junit4]   2> 2105402 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2105402 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2105416 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@37868ccb[collection1] realtime]
   [junit4]   2> 2105416 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2105452 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5280
   [junit4]   2> 2105467 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[560 (1604632533331345408), 561 (1604632533346025472), 562 (1604632533346025473), 563 (1604632533346025474), 564 (1604632533346025475), 565 (1604632533346025476), 566 (1604632533346025477), 567 (1604632533346025478), 568 (1604632533346025479), 569 (1604632533346025480), ... (20 adds)]} 0 14
   [junit4]   2> 2105668 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5496
   [junit4]   2> 2105670 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[580 (1604632533558886400), 581 (1604632533558886401), 582 (1604632533558886402), 583 (1604632533558886403), 584 (1604632533558886404), 585 (1604632533558886405), 586 (1604632533558886406), 587 (1604632533558886407), 588 (1604632533558886408), 589 (1604632533558886409), ... (20 adds)]} 0 1
   [junit4]   2> 2105873 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5701
   [junit4]   2> 2105875 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[600 (1604632533773844480), 601 (1604632533773844481), 602 (1604632533773844482), 603 (1604632533773844483), 604 (1604632533773844484), 605 (1604632533773844485), 606 (1604632533773844486), 607 (1604632533773844487), 608 (1604632533773844488), 609 (1604632533773844489), ... (20 adds)]} 0 1
   [junit4]   2> 2106078 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=5906
   [junit4]   2> 2106080 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[620 (1604632533988802560), 621 (1604632533988802561), 622 (1604632533988802562), 623 (1604632533988802563), 624 (1604632533988802564), 625 (1604632533988802565), 626 (1604632533988802566), 627 (1604632533988802567), 628 (1604632533988802568), 629 (1604632533989851136), ... (20 adds)]} 0 1
   [junit4]   2> 2106281 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6109
   [junit4]   2> 2106283 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[640 (1604632534201663488), 641 (1604632534201663489), 642 (1604632534201663490), 643 (1604632534201663491), 644 (1604632534201663492), 645 (1604632534201663493), 646 (1604632534201663494), 647 (1604632534201663495), 648 (1604632534201663496), 649 (1604632534201663497), ... (20 adds)]} 0 1
   [junit4]   2> 2106484 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6313
   [junit4]   2> 2106486 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[660 (1604632534414524416), 661 (1604632534414524417), 662 (1604632534414524418), 663 (1604632534414524419), 664 (1604632534414524420), 665 (1604632534414524421), 666 (1604632534414524422), 667 (1604632534414524423), 668 (1604632534415572992), 669 (1604632534415572993), ... (20 adds)]} 0 1
   [junit4]   2> 2106687 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6516
   [junit4]   2> 2106690 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[680 (1604632534627385344), 681 (1604632534627385345), 682 (1604632534627385346), 683 (1604632534627385347), 684 (1604632534627385348), 685 (1604632534627385349), 686 (1604632534628433920), 687 (1604632534628433921), 688 (1604632534628433922), 689 (1604632534628433923), ... (20 adds)]} 0 2
   [junit4]   2> 2106691 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2106691 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2106694 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f9d158[collection1] realtime]
   [junit4]   2> 2106695 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2106741 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6569
   [junit4]   2> 2106744 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[700 (1604632534682959872), 701 (1604632534684008448), 702 (1604632534685057024), 703 (1604632534685057025), 704 (1604632534685057026), 705 (1604632534685057027), 706 (1604632534685057028), 707 (1604632534685057029), 708 (1604632534685057030), 709 (1604632534685057031), ... (20 adds)]} 0 2
   [junit4]   2> 2106945 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6773
   [junit4]   2> 2106947 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[720 (1604632534896869376), 721 (1604632534896869377), 722 (1604632534897917952), 723 (1604632534897917953), 724 (1604632534897917954), 725 (1604632534897917955), 726 (1604632534897917956), 727 (1604632534897917957), 728 (1604632534897917958), 729 (1604632534897917959), ... (20 adds)]} 0 1
   [junit4]   2> 2107148 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=6976
   [junit4]   2> 2107150 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[740 (1604632535109730304), 741 (1604632535109730305), 742 (1604632535109730306), 743 (1604632535110778880), 744 (1604632535110778881), 745 (1604632535110778882), 746 (1604632535110778883), 747 (1604632535110778884), 748 (1604632535110778885), 749 (1604632535110778886), ... (20 adds)]} 0 1
   [junit4]   2> 2107351 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=7179
   [junit4]   2> 2107352 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[760 (1604632535322591232), 761 (1604632535322591233), 762 (1604632535322591234), 763 (1604632535322591235), 764 (1604632535322591236), 765 (1604632535322591237), 766 (1604632535323639808), 767 (1604632535323639809), 768 (1604632535323639810), 769 (1604632535323639811), ... (20 adds)]} 0 1
   [junit4]   2> 2107553 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=7382
   [junit4]   2> 2107555 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[780 (1604632535535452160), 781 (1604632535535452161), 782 (1604632535535452162), 783 (1604632535535452163), 784 (1604632535535452164), 785 (1604632535535452165), 786 (1604632535535452166), 787 (1604632535535452167), 788 (1604632535535452168), 789 (1604632535536500736), ... (20 adds)]} 0 1
   [junit4]   2> 2107756 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=7584
   [junit4]   2> 2107758 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[800 (1604632535748313088), 801 (1604632535748313089), 802 (1604632535748313090), 803 (1604632535748313091), 804 (1604632535748313092), 805 (1604632535748313093), 806 (1604632535748313094), 807 (1604632535748313095), 808 (1604632535748313096), 809 (1604632535748313097), ... (20 adds)]} 0 1
   [junit4]   2> 2107959 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=7787
   [junit4]   2> 2107961 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[820 (1604632535961174016), 821 (1604632535961174017), 822 (1604632535961174018), 823 (1604632535961174019), 824 (1604632535961174020), 825 (1604632535961174021), 826 (1604632535961174022), 827 (1604632535961174023), 828 (1604632535961174024), 829 (1604632535961174025), ... (20 adds)]} 0 1
   [junit4]   2> 2107962 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2107962 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2107966 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@108c952f[collection1] realtime]
   [junit4]   2> 2107966 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2108012 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=7840
   [junit4]   2> 2108014 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[840 (1604632536015699968), 841 (1604632536016748544), 842 (1604632536016748545), 843 (1604632536016748546), 844 (1604632536016748547), 845 (1604632536016748548), 846 (1604632536017797120), 847 (1604632536017797121), 848 (1604632536017797122), 849 (1604632536017797123), ... (20 adds)]} 0 2
   [junit4]   2> 2108216 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=8044
   [junit4]   2> 2108217 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[860 (1604632536229609472), 861 (1604632536229609473), 862 (1604632536229609474), 863 (1604632536229609475), 864 (1604632536229609476), 865 (1604632536229609477), 866 (1604632536230658048), 867 (1604632536230658049), 868 (1604632536230658050), 869 (1604632536230658051), ... (20 adds)]} 0 1
   [junit4]   2> 2108418 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=8247
   [junit4]   2> 2108420 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[880 (1604632536442470400), 881 (1604632536442470401), 882 (1604632536442470402), 883 (1604632536442470403), 884 (1604632536442470404), 885 (1604632536442470405), 886 (1604632536442470406), 887 (1604632536442470407), 888 (1604632536443518976), 889 (1604632536443518977), ... (20 adds)]} 0 1
   [junit4]   2> 2108621 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=8449
   [junit4]   2> 2108623 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[900 (1604632536655331328), 901 (1604632536655331329), 902 (1604632536655331330), 903 (1604632536655331331), 904 (1604632536655331332), 905 (1604632536655331333), 906 (1604632536655331334), 907 (1604632536655331335), 908 (1604632536655331336), 909 (1604632536655331337), ... (20 adds)]} 0 1
   [junit4]   2> 2108826 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=8654
   [junit4]   2> 2108828 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[920 (1604632536869240832), 921 (1604632536869240833), 922 (1604632536869240834), 923 (1604632536869240835), 924 (1604632536869240836), 925 (1604632536869240837), 926 (1604632536870289408), 927 (1604632536870289409), 928 (1604632536870289410), 929 (1604632536870289411), ... (20 adds)]} 0 1
   [junit4]   2> 2109028 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=8857
   [junit4]   2> 2109030 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[940 (1604632537082101760), 941 (1604632537082101761), 942 (1604632537082101762), 943 (1604632537082101763), 944 (1604632537082101764), 945 (1604632537082101765), 946 (1604632537082101766), 947 (1604632537082101767), 948 (1604632537082101768), 949 (1604632537082101769), ... (20 adds)]} 0 1
   [junit4]   2> 2109231 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=9059
   [junit4]   2> 2109235 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2109235 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2109238 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5424c066[collection1] realtime]
   [junit4]   2> 2109239 INFO  (commitScheduler-5673-thread-1) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2109240 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[960 (1604632537293914112), 961 (1604632537294962688), 962 (1604632537294962689), 963 (1604632537294962690), 964 (1604632537294962691), 965 (1604632537294962692), 966 (1604632537294962693), 967 (1604632537294962694), 968 (1604632537294962695), 969 (1604632537294962696), ... (20 adds)]} 0 8
   [junit4]   2> 2109441 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=9269
   [junit4]   2> 2109443 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[980 (1604632537514115072), 981 (1604632537514115073), 982 (1604632537515163648), 983 (1604632537515163649), 984 (1604632537515163650), 985 (1604632537515163651), 986 (1604632537515163652), 987 (1604632537515163653), 988 (1604632537515163654), 989 (1604632537515163655), ... (20 adds)]} 0 1
   [junit4]   2> 2109644 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=9472
   [junit4]   2> 2109644 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2109644 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2c871d8 commitCommandVersion:0
   [junit4]   2> 2109648 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@69a3d656[collection1] main]
   [junit4]   2> 2109650 INFO  (searcherExecutor-5670-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@69a3d656[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):c140) Uninverting(_1(8.0.0):c138) Uninverting(_2(8.0.0):c2) Uninverting(_3(8.0.0):c140) Uninverting(_4(8.0.0):c140) Uninverting(_5(8.0.0):c140) Uninverting(_6(8.0.0):c140) Uninverting(_7(8.0.0):c136) Uninverting(_8(8.0.0):c24)))}
   [junit4]   2> 2109650 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2109701 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&qt=&start=0&rows=20&version=2.2} hits=1000 status=0 QTime=9529
   [junit4]   2> 2109703 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending endToEndTest {seed=[3D1B3B37BA65E6C9:274D54015AD7C1CB]}
   [junit4]   2> 2109703 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2109703 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2050438469
   [junit4]   2> 2109703 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 2109703 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@51bcc7c3: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7c42a52d
   [junit4]   2> 2109710 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 2109710 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@61d1c900: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4fe896bb
   [junit4]   2> 2109714 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2109714 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6cfd51a5: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@369031fa
   [junit4]   2> 2109755 INFO  (coreCloseExecutor-5675-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@161e4a3a
   [junit4]   2> 2109755 INFO  (coreCloseExecutor-5675-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=161e4a3a
   [junit4]   2> 2109755 INFO  (coreCloseExecutor-5675-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4cef1b67: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@7686046
   [junit4]   2> 2109770 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting endToEndTest {seed=[3D1B3B37BA65E6C9:931BE8FD6E150AE7]}
   [junit4]   2> 2109770 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2109771 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2109840 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2109860 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2110312 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2110791 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2110791 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2110821 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2110862 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2110862 INFO  (TEST-MaxSizeAutoCommitTest.endToEndTest-seed#[3D1B3B37BA65E6C9]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2110867 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2110917 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2110961 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2111170 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2111190 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 2111214 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c55c5
   [junit4]   2> 2111214 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2111214 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.MaxSizeAutoCommitTest_3D1B3B37BA65E6C9-001/init-core-data-001/]
   [junit4]   2> 2111216 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2111221 WARN  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2111346 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2111346 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2111347 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2111347 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2111348 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=66.5576171875, floorSegmentMB=1.8759765625, forceMergeDeletesPctAllowed=8.949208482230127, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8701368330596397, reclaimDeletesWeight=3.661011280764526
   [junit4]   2> 2111348 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@723e7952[collection1] main]
   [junit4]   2> 2111349 WARN  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 2111349 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2111349 INFO  (coreLoadExecutor-5685-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using n

[...truncated too long message...]

e = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1398111468
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1398111468
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1398111468

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1033651815
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1033651815
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1033651815

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 69ms :: artifacts dl 8ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 170 minutes 30 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 86 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/86/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=30440024169784034, stage=STARTED, actionName='null', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024307906884, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024380214534, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024425235284, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440032298235734, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440032327056234, stage=SUCCEEDED, actionName='null', event={   "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",   "source":"index_size_trigger2",   "eventTime":30440020039431634,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440024160063934,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041704896084, stage=STARTED, actionName='null', event={   "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",   "source":"index_size_trigger2",   "eventTime":30440037513658484,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440041694060984,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041801755584, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",   "source":"index_size_trigger2",   "eventTime":30440037513658484,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440041694060984,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041898389534, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",   "source":"index_size_trigger2",   "eventTime":30440037513658484,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440041694060984,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041925142984, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",   "source":"index_size_trigger2",   "eventTime":30440037513658484,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30440041694060984,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=30440024169784034, stage=STARTED, actionName='null', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024307906884, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024380214534, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440024425235284, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440032298235734, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440032327056234, stage=SUCCEEDED, actionName='null', event={
  "id":"6c25097567f5d2Tcypqol625ax13sg8v9hjklyq7",
  "source":"index_size_trigger2",
  "eventTime":30440020039431634,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440024160063934,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041704896084, stage=STARTED, actionName='null', event={
  "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",
  "source":"index_size_trigger2",
  "eventTime":30440037513658484,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440041694060984,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041801755584, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",
  "source":"index_size_trigger2",
  "eventTime":30440037513658484,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440041694060984,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041898389534, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",
  "source":"index_size_trigger2",
  "eventTime":30440037513658484,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440041694060984,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30440041925142984, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6c250d86f38074Tcypqol625ax13sg8v9hjklyqa",
  "source":"index_size_trigger2",
  "eventTime":30440037513658484,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"base_url\":\"http://127.0.0.1:10000/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"base_url\":\"http://127.0.0.1:10001/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30440041694060984,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}] expected:<6> but was:<10>
	at __randomizedtesting.SeedInfo.seed([CF3DCBF92D2B5FB4:F6B372B902D4964A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:316)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([CF3DCBF92D2B5FB4:441A18286C2DF430]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([CF3DCBF92D2B5FB4:68D2D65AE266D0AC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:308)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12435 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_CF3DCBF92D2B5FB4-001/init-core-data-001
   [junit4]   2> 351644 WARN  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=105 numCloses=105
   [junit4]   2> 351644 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 351646 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 351646 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 351647 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_CF3DCBF92D2B5FB4-001/tempDir-001
   [junit4]   2> 351647 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 351647 INFO  (Thread-347) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 351647 INFO  (Thread-347) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 351650 ERROR (Thread-347) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 351747 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34652
   [junit4]   2> 351756 INFO  (zkConnectionManagerCallback-670-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 351760 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 351761 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 351762 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 351762 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 351763 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 351763 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30bf6b3a{/solr,null,AVAILABLE}
   [junit4]   2> 351764 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@204baa3f{HTTP/1.1,[http/1.1]}{127.0.0.1:36793}
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.e.j.s.Server Started @352171ms
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36793}
   [junit4]   2> 351765 ERROR (jetty-launcher-667-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 351765 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-28T12:13:12.236Z
   [junit4]   2> 351768 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 351768 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 351768 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 351768 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4afecf7f{/solr,null,AVAILABLE}
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5948f855{HTTP/1.1,[http/1.1]}{127.0.0.1:47237}
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.e.j.s.Server Started @352175ms
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47237}
   [junit4]   2> 351769 ERROR (jetty-launcher-667-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 351769 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-28T12:13:12.240Z
   [junit4]   2> 351796 INFO  (zkConnectionManagerCallback-672-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 351796 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 351802 INFO  (zkConnectionManagerCallback-674-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 351852 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 351870 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x107165bd9080002, likely client has closed socket
   [junit4]   2> 352182 INFO  (jetty-launcher-667-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34652/solr
   [junit4]   2> 352196 INFO  (zkConnectionManagerCallback-678-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352240 INFO  (zkConnectionManagerCallback-680-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352350 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 352351 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47237_solr
   [junit4]   2> 352352 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.Overseer Overseer (id=74052502613917700-127.0.0.1:47237_solr-n_0000000000) starting
   [junit4]   2> 352366 INFO  (zkConnectionManagerCallback-687-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352368 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34652/solr ready
   [junit4]   2> 352406 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47237_solr
   [junit4]   2> 352408 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 352410 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 352419 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 352419 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 352422 INFO  (OverseerStateUpdate-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 352457 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: []
   [junit4]   2> 352463 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 352463 INFO  (zkCallback-686-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 352463 DEBUG (OverseerAutoScalingTriggerThread-74052502613917700-127.0.0.1:47237_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 352477 DEBUG (ScheduledTrigger-740-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 352484 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 352525 INFO  (jetty-launcher-667-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34652/solr
   [junit4]   2> 352529 INFO  (zkConnectionManagerCallback-692-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352532 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47237.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352544 INFO  (zkConnectionManagerCallback-694-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352548 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47237.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352549 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47237.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352552 INFO  (jetty-launcher-667-thread-2) [n:127.0.0.1:47237_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_CF3DCBF92D2B5FB4-001/tempDir-001/node2/.
   [junit4]   2> 352553 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 352563 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 352566 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 352566 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36793_solr
   [junit4]   2> 352568 INFO  (zkCallback-686-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 352568 INFO  (zkCallback-679-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 352580 INFO  (zkCallback-693-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 352609 INFO  (zkConnectionManagerCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352611 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 352612 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34652/solr ready
   [junit4]   2> 352613 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 352636 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36793.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352648 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36793.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352649 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36793.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4a3f5692
   [junit4]   2> 352650 INFO  (jetty-launcher-667-thread-1) [n:127.0.0.1:36793_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_CF3DCBF92D2B5FB4-001/tempDir-001/node1/.
   [junit4]   2> 352683 INFO  (zkConnectionManagerCallback-704-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352687 INFO  (zkConnectionManagerCallback-709-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352689 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 352690 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34652/solr ready
   [junit4]   2> 352745 DEBUG (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 352748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 352748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 352748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 352749 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 352754 INFO  (SUITE-IndexSizeTriggerTest-seed#[CF3DCBF92D2B5FB4]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 352754 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 352757 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 352757 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 352757 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352778 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352800 DEBUG (ScheduledTrigger-756-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352820 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352840 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352860 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352881 DEBUG (ScheduledTrigger-756-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352901 DEBUG (ScheduledTrigger-756-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352921 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352941 DEBUG (ScheduledTrigger-756-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352961 DEBUG (ScheduledTrigger-756-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352982 DEBUG (ScheduledTrigger-756-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 352985 INFO  (TEST-IndexSizeTriggerTest.testTrigger-seed#[CF3DCBF92D2B5FB4]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
   [junit4]   2> 353002 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353023 DEBUG (ScheduledTrigger-756-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353044 DEBUG (ScheduledTrigger-756-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353061 DEBUG (simCloudManagerPool-755-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testTrigger_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testTrigger_collection"}}
   [junit4]   2> 353065 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353075 DEBUG (simCloudManagerPool-755-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testTrigger_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testTrigger_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testTrigger_collection"}}
   [junit4]   2> 353085 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353105 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353125 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353145 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353166 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353186 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353206 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353226 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353246 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353267 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353287 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353307 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353327 DEBUG (ScheduledTrigger-756-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353337 INFO  (TEST-IndexSizeTriggerTest.testTrigger-seed#[CF3DCBF92D2B5FB4]) [    ] o.a.s.c.a.IndexSizeTriggerTest #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[testTrigger_collection]
   [junit4]   2> DocCollection(testTrigger_collection//clusterstate.json/30)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testTrigger_collection_shard2_replica_n3",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":15740,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.4659017324447632E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":11},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testTrigger_collection_shard2_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":15740,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.4659017324447632E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":11}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testTrigger_collection_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":17240,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.605600118637085E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":14},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testTrigger_collection_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":17240,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.605600118637085E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":14}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> ## Max replicas per node:	2
   [junit4]   2> ## Min replicas per node:	2
   [junit4]   2> ## Total replicas:		4
   [junit4]   2> ## * testTrigger_collection		4
   [junit4]   2> ##		- active           4
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- update            26
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> 
   [junit4]   2> 353338 DEBUG (TEST-IndexSizeTriggerTest.testTrigger-seed#[CF3DCBF92D2B5FB4]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 353338 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 353338 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 353339 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 353339 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 353439 INFO  (TEST-IndexSizeTriggerTest.testTrigger-seed#[CF3DCBF92D2B5FB4]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
   [junit4]   2> 353469 INFO  (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[CF3DCBF92D2B5FB4]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMixedBounds
   [junit4]   2> 353478 DEBUG (ScheduledTrigger-740-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 353504 DEBUG (simCloudManagerPool-755-thread-39) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMixedBounds_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testMixedBounds_collection"}}
   [junit4]   2> 353504 DEBUG (simCloudManagerPool-755-thread-40) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMixedBounds_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testMixedBounds_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testMixedBounds_collection"}}
   [junit4]   2> 353638 DEBUG (simCloudManagerPool-755-thread-53) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 353639 DEBUG (simCloudManagerPool-755-thread-53) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 353640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 353640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 353640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 353640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 353643 DEBUG (simCloudManagerPool-755-thread-54) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 353643 DEBUG (simCloudManagerPool-755-thread-54) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 353644 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 353644 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 353644 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 353645 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 353646 DEBUG (simCloudManagerPool-755-thread-55) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 353646 DEBUG (simCloudManagerPool-755-thread-55) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 353647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 353647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 353647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 353647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 353667 DEBUG (simCloudManagerPool-755-thread-56) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 353668 DEBUG (simCloudManagerPool-755-thread-56) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 353668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 353668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 353668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 353668 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 353752 DEBUG (ScheduledTrigger-756-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6c24f68f55b62cTcypqol625ax13sg8v9hjklypy",
   [junit4]   2>   "source":"index_size_trigger4",
   [junit4]   2>   "eventTime":30439938870064684,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testMixedBounds_collection":[
   [junit4]   2>         "{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":505,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":262740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":505,\n    \"__bytes__\":262740,\n    \"core\":\"testMixedBounds_collection_shard2_replica_n3\",\n    \"__docs__\":505,\n    \"violationType\":\"aboveBytes\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.446956932544708E-4,\n    \"shard\":\"shard2\",\n    \"collection\":\"testMixedBounds_collection\"}}",
   [junit4]   2>         "{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":495,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":257740,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":495,\n    \"__bytes__\":257740,\n    \"core\":\"testMixedBounds_collection_shard1_replica_

[...truncated too long message...]

ile = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1208621160
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1208621160
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1208621160

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null292363920
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null292363920
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null292363920

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 87ms :: artifacts dl 13ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 170 minutes 38 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 85 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/85/

6 tests failed.
FAILED:  org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection

Error Message:
Timeout waiting for new leader null Live Nodes: [127.0.0.1:40061_solr, 127.0.0.1:42415_solr, 127.0.0.1:47051_solr] Last available state: DocCollection(collection1//collections/collection1/state.json/14)={   "pullReplicas":"0",   "replicationFactor":"3",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node62":{           "core":"collection1_shard1_replica_n61",           "base_url":"https://127.0.0.1:42098/solr",           "node_name":"127.0.0.1:42098_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"},         "core_node64":{           "core":"collection1_shard1_replica_n63",           "base_url":"https://127.0.0.1:42415/solr",           "node_name":"127.0.0.1:42415_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node66":{           "core":"collection1_shard1_replica_n65",           "base_url":"https://127.0.0.1:47051/solr",           "node_name":"127.0.0.1:47051_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"3",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Timeout waiting for new leader
null
Live Nodes: [127.0.0.1:40061_solr, 127.0.0.1:42415_solr, 127.0.0.1:47051_solr]
Last available state: DocCollection(collection1//collections/collection1/state.json/14)={
  "pullReplicas":"0",
  "replicationFactor":"3",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node62":{
          "core":"collection1_shard1_replica_n61",
          "base_url":"https://127.0.0.1:42098/solr",
          "node_name":"127.0.0.1:42098_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node64":{
          "core":"collection1_shard1_replica_n63",
          "base_url":"https://127.0.0.1:42415/solr",
          "node_name":"127.0.0.1:42415_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node66":{
          "core":"collection1_shard1_replica_n65",
          "base_url":"https://127.0.0.1:47051/solr",
          "node_name":"127.0.0.1:47051_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"3",
  "tlogReplicas":"0"}
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:679EBD311170F14B]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
	at org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection(LeaderVoteWaitTimeoutTest.java:189)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:44A5725A92366EE5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi

Error Message:
expected:<2> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<0>
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:98AB5A3E08C2277A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi(AutoScalingHandlerTest.java:845)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:AC4997094AFFB64C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode

Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=8},status={state=notfound,msg=Did not find [search_rate_trigger3/6bced90b5eb068T458wd97sfj1nhkezbh3g0jva1/0] in any tasks queue}}

Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=8},status={state=notfound,msg=Did not find [search_rate_trigger3/6bced90b5eb068T458wd97sfj1nhkezbh3g0jva1/0] in any tasks queue}}
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:ED106F09E4FA4A1C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:670)
	at java.util.ArrayList.forEach(ArrayList.java:1257)
	at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:662)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:686DBC281C7D4A79]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:308)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12367 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestLargeCluster_CF82A18BD330C561-001/init-core-data-001
   [junit4]   2> 111606 WARN  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 111606 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 111608 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 111610 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 111678 DEBUG (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 111687 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 111687 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 111689 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 111703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 111703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 111724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 111727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 111730 DEBUG (ScheduledTrigger-130-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111740 INFO  (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
   [junit4]   2> 111740 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 111740 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 111742 INFO  (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 111746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 111746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 111765 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 111765 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 111775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 111775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 111746 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 111777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111777 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 111785 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111806 DEBUG (ScheduledTrigger-135-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111826 DEBUG (ScheduledTrigger-135-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111847 DEBUG (ScheduledTrigger-135-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111916 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111936 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111952 DEBUG (simCloudManagerPool-134-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 111953 DEBUG (simCloudManagerPool-134-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 111957 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111960 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 111961 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111961 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 111974 DEBUG (simCloudManagerPool-134-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 111974 DEBUG (simCloudManagerPool-134-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 111975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 111975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 111975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111975 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 112036 DEBUG (simCloudManagerPool-134-thread-5) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 112037 DEBUG (simCloudManagerPool-134-thread-5) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 112038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 112038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 112039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=start, class=org.apache.solr.cloud.autoscaling.sim.TestLargeCluster$StartTriggerAction}, {name=compute, class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestLargeCluster$FinishTriggerAction}], enabled=true}
   [junit4]   2> 112039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 112040 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 112040 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112040 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112061 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112061 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112081 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112081 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112102 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112103 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112124 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112124 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112144 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112144 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112166 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112166 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112186 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112187 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112207 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112207 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112229 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112229 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112249 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112249 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112270 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112270 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112290 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112290 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112311 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112311 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112331 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112331 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112351 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112352 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112459 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112459 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112480 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112480 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112500 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112500 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112521 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112521 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112542 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112542 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112563 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112563 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112584 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112584 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112604 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112604 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112625 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112625 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112646 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112646 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112666 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112666 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112687 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112687 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112707 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112707 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112728 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112728 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112748 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112748 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112769 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112769 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112789 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112789 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112810 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112810 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112830 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112830 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112851 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112851 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112871 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112871 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112938 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112938 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112958 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112958 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 112978 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 112979 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113005 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113006 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113026 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113026 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113046 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113047 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113067 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113067 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113103 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113103 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113123 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113123 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113144 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113144 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113164 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113164 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113184 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113185 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113205 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113205 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113226 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113226 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113246 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113246 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113267 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113267 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113287 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113287 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113307 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113308 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113328 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113328 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113348 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113349 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113369 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113369 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113458 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113458 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113522 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113522 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113542 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113542 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113563 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113563 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113583 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113583 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113604 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113604 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113624 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113624 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113644 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113644 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113665 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113665 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113685 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113685 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113705 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113705 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113726 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113726 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113746 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113746 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113767 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113767 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113787 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113787 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113807 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113807 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113828 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113828 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113850 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113850 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113870 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113871 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113891 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113891 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113911 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113911 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113934 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113934 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113954 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113954 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113975 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113975 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 113996 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 113996 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114016 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114016 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114037 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114037 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114057 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114057 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114078 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114078 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114098 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114098 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114119 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114119 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114139 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114139 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114160 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114160 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114180 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114180 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114200 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114201 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114223 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114223 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114244 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114244 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114265 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114265 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114286 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114286 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114306 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114306 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114326 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114327 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114347 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114347 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114367 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114367 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114388 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114388 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114408 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114408 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114428 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114428 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114449 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114449 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114469 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114469 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114489 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114489 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114510 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114510 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114530 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114530 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114550 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114550 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114571 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114571 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114591 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114591 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114611 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114612 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114632 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114632 DEBUG (ScheduledTrigger-135-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114653 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114653 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114674 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114674 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114694 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114694 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114715 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114715 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114735 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114735 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114756 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114756 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114776 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114776 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114796 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114796 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114817 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114817 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114837 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114837 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114857 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114857 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114878 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114878 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114898 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114898 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114919 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114919 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114939 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114939 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114959 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114960 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 114980 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 114980 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115000 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115000 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115021 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115021 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115041 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115041 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115062 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115062 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115082 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115082 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115103 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115103 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115123 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115123 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115144 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115144 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115164 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115165 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115185 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115185 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115205 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115206 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115226 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115226 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115246 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115247 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115267 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115267 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115287 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115287 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115308 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115308 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115328 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115328 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115349 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115349 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115369 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115369 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115390 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115390 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115465 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115465 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115485 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115486 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115506 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115506 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115526 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115526 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115547 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115547 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115567 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115567 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115588 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115588 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115608 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115608 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115628 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115628 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115649 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115649 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115669 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115669 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115690 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115690 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115710 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115710 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115730 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115730 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115751 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115751 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115771 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115771 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 115792 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
   [junit4]   2> 115792 DEBUG (ScheduledTrigger-135-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found l

[...truncated too long message...]

1, tag=781fb793
   [junit4]   2> 3623459 INFO  (coreCloseExecutor-10835-thread-1) [n:127.0.0.1:57723_solr c:belowRate_collection s:shard1 r:core_node3 x:belowRate_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@116d4db0: rootName = solr_57723, domain = solr.core.belowRate_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.belowRate_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@63fe3c63
   [junit4]   2> 3623461 INFO  (jetty-closer-9923-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3c24f50{/solr,null,UNAVAILABLE}
   [junit4]   2> 3623461 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10710b7065c0007, likely client has closed socket
   [junit4]   2> 3623461 INFO  (zkCallback-9954-thread-4) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57723_solr
   [junit4]   2> 3623462 INFO  (jetty-closer-9923-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3b7bf14{/solr,null,UNAVAILABLE}
   [junit4]   2> 3623470 INFO  (jetty-closer-9923-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3623482 INFO  (jetty-closer-9923-thread-3) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3623482 INFO  (jetty-closer-9923-thread-4) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3623490 INFO  (coreCloseExecutor-10835-thread-1) [n:127.0.0.1:57723_solr c:belowRate_collection s:shard1 r:core_node3 x:belowRate_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.belowRate_collection.shard1.leader, tag=781fb793
   [junit4]   2> 3623758 INFO  (jetty-closer-9923-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3894b9f9{/solr,null,UNAVAILABLE}
   [junit4]   2> 3623758 INFO  (jetty-closer-9923-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3623762 ERROR (SUITE-SearchRateTriggerIntegrationTest-seed#[CF82A18BD330C561]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3623763 INFO  (SUITE-SearchRateTriggerIntegrationTest-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39479 39479
   [junit4]   2> 3623878 INFO  (Thread-6303) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39479 39479
   [junit4]   2> 3623879 WARN  (Thread-6303) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	53	/solr/collections/aboveRate_collection/terms/shard1
   [junit4]   2> 	44	/solr/collections/belowRate_collection/terms/shard1
   [junit4]   2> 	44	/solr/collections/deleteNode_collection/terms/shard1
   [junit4]   2> 	15	/solr/configs/conf
   [junit4]   2> 	14	/solr/clusterprops.json
   [junit4]   2> 	14	/solr/aliases.json
   [junit4]   2> 	5	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	27	/solr/collections/belowRate_collection/state.json
   [junit4]   2> 	25	/solr/collections/aboveRate_collection/state.json
   [junit4]   2> 	23	/solr/collections/deleteNode_collection/state.json
   [junit4]   2> 	14	/solr/clusterstate.json
   [junit4]   2> 	6	/solr/aliases.json
   [junit4]   2> 	3	/solr/collections/deleteNode_collection/leader_elect/shard1/election/74046297146654739-core_node3-n_0000000000
   [junit4]   2> 	2	/solr/collections/aboveRate_collection/leader_elect/shard1/election/74046297146654730-core_node3-n_0000000001
   [junit4]   2> 	2	/solr/collections/belowRate_collection/leader_elect/shard1/election/74046297146654739-core_node4-n_0000000001
   [junit4]   2> 	2	/solr/overseer_elect/election/74046297146654731-127.0.0.1:57723_solr-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	25	/solr/collections
   [junit4]   2> 	14	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_CF82A18BD330C561-001
   [junit4]   2> Jun 27, 2018 9:57:41 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=132, maxMBSortInHeap=5.894151473970627, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@19fcb127), locale=ar-MA, timezone=Asia/Tashkent
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=168775456,total=518520832
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudNestedDocsSort, LargeFieldTest, DistribDocExpirationUpdateProcessorTest, TestLargeCluster, TestDynamicURP, InfixSuggestersTest, TestPushWriter, TestInitParams, SolrIndexSplitterTest, DirectUpdateHandlerTest, TestConfigSetsAPI, TestReloadDeadlock, TolerantUpdateProcessorTest, TestRestoreCore, QueryParsingTest, TestStreamBody, HdfsTlogReplayBufferedWhileIndexingTest, SpatialRPTFieldTypeTest, TestPolicyCloud, SolrCoreTest, TestMinMaxOnMultiValuedField, TestCoreContainer, TestManagedResource, TestFieldCacheSort, CoreAdminHandlerTest, ResponseLogComponentTest, CollectionTooManyReplicasTest, TestScoreJoinQPScore, TestTolerantUpdateProcessorRandomCloud, TestStandardQParsers, TestSerializedLuceneMatchVersion, Tagger2Test, TestLMDirichletSimilarityFactory, SSLMigrationTest, SuggesterWFSTTest, TestHttpShardHandlerFactory, TestSha256AuthenticationProvider, HdfsDirectoryFactoryTest, TestBulkSchemaAPI, SolrCmdDistributorTest, TestDistributedMissingSort, DistanceFunctionTest, OverseerCollectionConfigSetProcessorTest, TestRandomDVFaceting, TestSimpleTextCodec, ConnectionReuseTest, PreAnalyzedFieldTest, MissingSegmentRecoveryTest, DocValuesTest, StressHdfsTest, TestSolrCoreProperties, SolrJmxReporterTest, TestUseDocValuesAsStored2, TestGroupingSearch, TestDistribStateManager, BasicDistributedZk2Test, LegacyCloudClusterPropTest, RemoteQueryErrorTest, SuggesterFSTTest, DistributedFacetPivotLongTailTest, PeerSyncWithLeaderAndIndexFingerprintCachingTest, ShardSplitTest, SpellCheckCollatorTest, AnalysisAfterCoreReloadTest, TestConfigReload, ZkSolrClientTest, TestInPlaceUpdatesStandalone, DistributedQueryComponentOptimizationTest, DistributedVersionInfoTest, TestBlendedInfixSuggestions, TestEmbeddedSolrServerConstructors, CustomHighlightComponentTest, ConcurrentDeleteAndCreateCollectionTest, TestSegmentSorting, TestCollectionsAPIViaSolrCloudCluster, HighlighterMaxOffsetTest, PingRequestHandlerTest, TestNumericTerms64, SearchHandlerTest, SyncSliceTest, IgnoreLargeDocumentProcessorFactoryTest, IndexSizeTriggerTest, DOMUtilTest, TestCollapseQParserPlugin, OverseerStatusTest, TestFastOutputStream, TestFoldingMultitermQuery, TokenizerChainTest, AtomicUpdatesTest, CdcrVersionReplicationTest, BadIndexSchemaTest, PathHierarchyTokenizerFactoryTest, V2ApiIntegrationTest, ZkFailoverTest, SolrCLIZkUtilsTest, QueryResultKeyTest, TestRTGBase, CursorPagingTest, TestReplicationHandler, TestSolrConfigHandlerCloud, HdfsNNFailoverTest, HdfsRestartWhileUpdatingTest, SampleTest, TestSolrDeletionPolicy2, TestCrossCoreJoin, SolrMetricManagerTest, TestIndexSearcher, TestNamedUpdateProcessors, TestLuceneMatchVersion, RulesTest, BlockCacheTest, TestCloudDeleteByQuery, SimpleFacetsTest, TestCryptoKeys, TestJsonFacetsWithNestedObjects, DocExpirationUpdateProcessorFactoryTest, TestCustomStream, TestSurroundQueryParser, TimeZoneUtilsTest, TestXmlQParser, OpenExchangeRatesOrgProviderTest, CoreMergeIndexesAdminHandlerTest, TestCodecSupport, TestRTimerTree, TestPullReplicaErrorHandling, TestMergePolicyConfig, CopyFieldTest, TestDocTermOrdsUninvertLimit, FullSolrCloudDistribCmdsTest, NodeLostTriggerTest, ShardRoutingTest, TestFieldSortValues, MultiThreadedOCPTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, ReplicationFactorTest, TestCloudPseudoReturnFields, TestCloudRecovery, TestHashPartitioner, TestPrepRecovery, TestSolrCloudWithSecureImpersonation, TestStressLiveNodes, TestZkChroot, TlogReplayBufferedWhileIndexingTest, TriLevelCompositeIdRoutingTest, ZkControllerTest, TestHdfsCloudBackupRestore, TestLocalFSCloudBackupRestore, TestRequestStatusCollectionAPI, ExecutePlanActionTest, HttpTriggerListenerTest, ScheduledMaintenanceTriggerTest, SearchRateTriggerIntegrationTest]
   [junit4] Completed [568/820 (6!)] on J0 in 157.90s, 3 tests, 1 failure <<< FAILURES!

[...truncated 51196 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/build.xml:642: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build.xml:670: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build.xml:639: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/common-build.xml:2573: Failed to load /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/dev-tools/doap/solr.rdf

Total time: 171 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 84 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/84/

2 tests failed.
FAILED:  org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic

Error Message:
{} expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: {} expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([60DC9B6165AE25D3:CB268674BA72A3FD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testUpdateStream

Error Message:
Error from server at http://127.0.0.1:33216/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000043

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33216/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000043
	at __randomizedtesting.SeedInfo.seed([3A3F22E5E426824E:F278E69FED2E7F1E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testUpdateStream(StreamDecoratorTest.java:2394)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12780 lines...]
   [junit4] Suite: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest
   [junit4]   2> 1111477 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_60DC9B6165AE25D3-001/init-core-data-001
   [junit4]   2> 1111478 WARN  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=27 numCloses=27
   [junit4]   2> 1111478 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1111479 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1111552 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[60DC9B6165AE25D3]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 1111619 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[60DC9B6165AE25D3]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasic
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrRrdBackendFactoryTest -Dtests.method=testBasic -Dtests.seed=60DC9B6165AE25D3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=zh-CN -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.16s J2 | SolrRrdBackendFactoryTest.testBasic <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: {} expected:<1> but was:<0>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([60DC9B6165AE25D3:CB268674BA72A3FD]:0)
   [junit4]    > 	at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_60DC9B6165AE25D3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1686, maxMBSortInHeap=6.46385235553262, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@740cfbbf), locale=zh-CN, timezone=Africa/Ouagadougou
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=240570696,total=527433728
   [junit4]   2> NOTE: All tests run in this JVM: [TestZkAclsWithHadoopAuth, BasicAuthStandaloneTest, TestTolerantUpdateProcessorRandomCloud, TestPullReplicaErrorHandling, TestSortableTextField, TestQueryTypes, TestInfoStreamLogging, SolrTestCaseJ4Test, TestSafeXMLParsing, TestManagedSynonymGraphFilterFactory, UnloadDistributedZkTest, TestPayloadCheckQParserPlugin, TestCloudPseudoReturnFields, DeleteInactiveReplicaTest, TimeZoneUtilsTest, CollectionTooManyReplicasTest, SearchRateTriggerIntegrationTest, TestUniqueKeyFieldResource, HdfsBasicDistributedZkTest, TestUpdate, PreAnalyzedFieldManagedSchemaCloudTest, AssignBackwardCompatibilityTest, PeerSyncWithIndexFingerprintCachingTest, BufferStoreTest, TestReqParamsAPI, TestNumericTokenStream, SpellingQueryConverterTest, TestCodecSupport, SolrCmdDistributorTest, TestSolrConfigHandler, TestManagedSchema, RecoveryZkTest, DirectSolrSpellCheckerTest, SearchHandlerTest, ChaosMonkeySafeLeaderWithPullReplicasTest, SolrRrdBackendFactoryTest]
   [junit4] Completed [123/821 (1!)] on J2 in 0.46s, 1 test, 1 failure <<< FAILURES!

[...truncated 3289 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.StreamDecoratorTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/init-core-data-001
   [junit4]   2> 14693 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 14695 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 14695 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 14699 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001
   [junit4]   2> 14714 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 14719 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 14719 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 14819 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:35628
   [junit4]   2> 14834 ERROR (Thread-2) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 14934 INFO  (zkConnectionManagerCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15014 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102dda6e7570000, likely client has closed socket
   [junit4]   2> 15430 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15437 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15437 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15444 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15549 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15549 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15553 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15588 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e29c6dc{/solr,null,AVAILABLE}
   [junit4]   2> 15589 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3fe18418{/solr,null,AVAILABLE}
   [junit4]   2> 15633 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15633 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15634 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 15645 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53cac00c{/solr,null,AVAILABLE}
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15873 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3521fd2a{/solr,null,AVAILABLE}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4e218bb6{HTTP/1.1,[http/1.1]}{127.0.0.1:33216}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5bc5beae{HTTP/1.1,[http/1.1]}{127.0.0.1:40736}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.Server Started @15921ms
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.Server Started @15921ms
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40736}
   [junit4]   2> 15884 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2bffa70{HTTP/1.1,[http/1.1]}{127.0.0.1:35748}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server Started @15928ms
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35748}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@24b796ea{HTTP/1.1,[http/1.1]}{127.0.0.1:34830}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server Started @15929ms
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34830}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33216}
   [junit4]   2> 15921 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
   [junit4]   2> 15925 ERROR (jetty-launcher-3-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15925 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.744Z
   [junit4]   2> 15927 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 15931 ERROR (jetty-launcher-3-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.749Z
   [junit4]   2> 15941 ERROR (jetty-launcher-3-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.759Z
   [junit4]   2> 15946 ERROR (jetty-launcher-3-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.764Z
   [junit4]   2> 15977 INFO  (zkConnectionManagerCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15993 INFO  (zkConnectionManagerCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16017 INFO  (zkConnectionManagerCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16019 INFO  (zkConnectionManagerCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16019 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16019 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16025 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16027 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16100 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102dda6e7570004, likely client has closed socket
   [junit4]   2> 16474 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16539 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16588 INFO  (zkConnectionManagerCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16629 INFO  (zkConnectionManagerCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16774 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16791 INFO  (zkConnectionManagerCallback-26-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16809 INFO  (zkConnectionManagerCallback-25-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16826 INFO  (zkConnectionManagerCallback-30-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16905 INFO  (zkConnectionManagerCallback-32-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17039 WARN  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17059 WARN  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17063 WARN  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17099 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17102 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34830_solr
   [junit4]   2> 17103 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17105 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.Overseer Overseer (id=72864252906897418-127.0.0.1:34830_solr-n_0000000000) starting
   [junit4]   2> 17110 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17114 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33216_solr
   [junit4]   2> 17123 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35748_solr
   [junit4]   2> 17172 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17173 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17173 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17213 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 17245 INFO  (zkConnectionManagerCallback-42-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17325 INFO  (zkConnectionManagerCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17328 INFO  (zkConnectionManagerCallback-54-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17335 INFO  (zkConnectionManagerCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17343 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17344 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17352 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17357 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17358 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17360 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17371 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17352 INFO  (zkConnectionManagerCallback-47-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17399 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17489 WARN  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17490 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17494 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17500 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 17500 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40736_solr
   [junit4]   2> 17526 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17541 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17541 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17541 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17558 INFO  (zkCallback-53-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17558 INFO  (zkCallback-46-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17558 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17565 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/.
   [junit4]   2> 17566 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17567 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17571 INFO  (zkCallback-52-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17584 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/.
   [junit4]   2> 17655 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 17655 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34830_solr
   [junit4]   2> 17662 INFO  (zkConnectionManagerCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17678 INFO  (zkCallback-53-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17679 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17679 INFO  (zkCallback-52-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17680 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17681 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17681 INFO  (zkCallback-46-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17685 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17692 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 17695 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17709 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17857 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17915 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17980 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18000 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18000 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18006 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node3/.
   [junit4]   2> 18035 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18036 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18084 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/.
   [junit4]   2> 18313 INFO  (zkConnectionManagerCallback-72-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 18349 INFO  (zkConnectionManagerCallback-77-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 18351 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 18353 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 18752 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 18801 INFO  (OverseerThreadFactory-49-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 18967 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35748/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 18993 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:34830/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 19201 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr    x:collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 19202 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr    x:collection1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 19207 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr    x:collection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 20268 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 20274 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 20302 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema [collection1_shard2_replica_n2] Schema name=test
   [junit4]   2> 20313 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 20510 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20533 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20533 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20543 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 20543 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 20544 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 20554 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 20572 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 20572 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 20573 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 20573 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from collection collection1, trusted=true
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/./collection1_shard1_replica_n1/data/]
   [junit4]   2> 20579 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 20580 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 20580 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/collection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/./collection1_shard2_replica_n2/data/]
   [junit4]   2> 20742 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 20742 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 20744 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 20744 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 20747 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b5419a3[collection1_shard1_replica_n1] main]
   [junit4]   2> 20754 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 20754 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 20756 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 20756 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 20758 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@64c2290f[collection1_shard2_replica_n2] main]
   [junit4]   2> 20760 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 20761 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 20763 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 20763 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 20764 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 20767 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291125664808960
   [junit4]   2> 20779 INFO  (searcherExecutor-58-thread-1-processing-n:127.0.0.1:34830_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2] Registered new searcher Searcher@64c2290f[collection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 20780 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 20781 INFO  (searcherExecutor-57-thread-1-processing-n:127.0.0.1:35748_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@7b5419a3[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 20783 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291125680537600
   [junit4]   2> 20833 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 20850 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 20884 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 20884 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 20885 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/
   [junit4]   2> 20907 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 20910 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 20910 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 20952 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 20954 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/ shard2
   [junit4]   2> 21121 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 21121 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 21129 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1927
   [junit4]   2> 21138 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1940
   [junit4]   2> 21202 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 21232 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 21234 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 22205 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3455
   [junit4]   2> 22214 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):90
   [junit4]   2> 22244 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 22314 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testParallelDaemonCommitStream
   [junit4]   2> 22461 INFO  (qtp1190851269-72) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=1}, version=1}
   [junit4]   2> 22462 INFO  (qtp1190851269-72) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1604291127428513792)} 0 14
   [junit4]   2> 22483 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291127464165376,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 22483 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 22487 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291127468359680,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 22487 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 22494 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 22494 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 11
   [junit4]   2> 22499 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 22499 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 12
   [junit4]   2> 22503 INFO  (qtp2089967312-64) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=1}, version=1}
   [junit4]   2> 22504 INFO  (qtp2089967312-64) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=collection1:4&commit=true&wt=javabin&version=2}{deleteByQuery=*:* (-1604291127403347968),commit=} 0 172
   [junit4]   2> 22506 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=parallelDestinationCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 22519 INFO  (OverseerCollectionConfigSetProcessor-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 22519 INFO  (OverseerThreadFactory-49-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection parallelDestinationCollection1
   [junit4]   2> 22743 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"parallelDestinationCollection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"parallelDestinationCollection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33216/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 22747 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"parallelDestinationCollection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"parallelDestinationCollection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35748/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 22950 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr    x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 22951 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr    x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 22954 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr    x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 23993 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 23995 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 24034 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema [parallelDestinationCollection1_shard2_replica_n2] Schema name=test
   [junit4]   2> 24043 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [parallelDestinationCollection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 24188 WARN  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 24188 WARN  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 24188 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'parallelDestinationCollection1_shard2_replica_n2' using configuration from collection parallelDestinationCollection1, trusted=true
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.core.parallelDestinationCollection1.shard2.replica_n2' (registry 'solr.core.parallelDestinationCollection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 24190 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore [[parallelDestinationCollection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/parallelDestinationCollection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/./parallelDestinationCollection1_shard2_replica_n2/data/]
   [junit4]   2> 24339 WARN  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 24339 WARN  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 24339 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 24339 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'parallelDestinationCollection1_shard1_replica_n1' using configuration from collection parallelDestinationCollection1, trusted=true
   [junit4]   2> 24361 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.core.parallelDestinationCollection1.shard1.replica_n1' (registry 'solr.core.parallelDestinationCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 24361 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 24362 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[parallelDestinationCollection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/parallelDestinationCollection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/./parallelDestinationCollection1_shard1_replica_n1/data/]
   [junit4]   2> 24364 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 24364 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 24366 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 24366 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 24369 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a49dcb7[parallelDestinationCollection1_shard2_replica_n2] main]
   [junit4]   2> 24371 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 24385 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 24386 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 24417 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291129492111360
   [junit4]   2> 24435 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/parallelDestinationCollection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 24440 INFO  (searcherExecutor-67-thread-1-processing-n:127.0.0.1:35748_solr x:parallelDestinationCollection1_shard2_replica_n2 c:parallelDestinationCollection1 s:shard2 r:core_node4) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore [parallelDestinationCollection1_shard2_replica_n2] Registered new searcher Searcher@5a49dcb7[parallelDestinationCollection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 24446 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 24446 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 24451 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/ shard2
   [junit4]   2> 24559 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 24559 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 24560 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 24560 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 24562 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@85d3657[parallelDestinationCollection1_shard1_replica_n1] main]
   [junit4]   2> 24564 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 24564 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 24565 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 24565 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291129647300608
   [junit4]   2> 24568 INFO  (searcherExecutor-69-thread-1-processing-n:127.0.0.1:33216_solr x:parallelDestinationCollection1_shard1_replica_n1 c:parallelDestinationCollection1 s:shard1 r:core_node3) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore [parallelDestinationCollection1_shard1_replica_n1] Registered new searcher Searcher@85d3657[parallelDestinationCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 24577 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/parallelDestinationCollection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 24595 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/ shard1
   [junit4]   2> 24605 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 24608 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1658
   [junit4]   2> 24712 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 24751 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 24755 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1804
   [junit4]   2> 24760 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 24855 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 24855 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 25760 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=parallelDestinationCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3254
   [junit4]   2> 25761 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: parallelDestinationCollection1 failOnTimeout:true timeout (sec):90
   [junit4]   2> 25762 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: parallelDestinationCollection1
   [junit4]   2> 25805 INFO  (qtp2089967312-68) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=collection1:4&wt=javabin&version=2}{add=[2 (1604291130911883264), 3 (1604291130947534848)]} 0 36
   [junit4]   2> 25807 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection1:4&wt=javabin&version=2}{add=[0 (1604291130908737536), 4 (1604291130947534848), 1 (1604291130948583424)]} 0 40
   [junit4]   2> 25814 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291130956972032,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25814 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@14f261fa commitCommandVersion:1604291130956972032
   [junit4]   2> 25827 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291130970603520,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25830 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5b44ef3c commitCommandVersion:1604291130970603520
   [junit4]   2> 25901 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@712c34a1[collection1_shard1_replica_n1] main]
   [junit4]   2> 25901 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@301d760b[collection1_shard2_replica_n2] main]
   [junit4]   2> 25910 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 25913 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 25921 INFO  (searcherExecutor-57-thread-1-processing-n:127.0.0.1:35748_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.S

[...truncated too long message...]

-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 21ms :: artifacts dl 4ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 219 minutes 44 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 83 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/83/

4 tests failed.
FAILED:  org.apache.solr.cloud.AddReplicaTest.test

Error Message:
core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:43801/solr","node_name":"127.0.0.1:43801_solr","state":"active","type":"NRT","force_set_state":"false"}

Stack Trace:
java.lang.AssertionError: core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:43801/solr","node_name":"127.0.0.1:43801_solr","state":"active","type":"NRT","force_set_state":"false"}
	at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:84572D44A92B366C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.AddReplicaTest.test(AddReplicaTest.java:85)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
	at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:5FBA502EE5C6CE6E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:427)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=30199874855567514, stage=STARTED, actionName='null', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876032314964, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876087984514, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876110009064, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883381681164, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883412704014, stage=SUCCEEDED, actionName='null', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893725932164, stage=STARTED, actionName='null', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893847188214, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893992554314, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199894023424714, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=30199874855567514, stage=STARTED, actionName='null', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876032314964, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876087984514, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876110009064, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883381681164, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883412704014, stage=SUCCEEDED, actionName='null', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893725932164, stage=STARTED, actionName='null', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893847188214, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893992554314, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199894023424714, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}] expected:<6> but was:<10>
	at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:358DABDE2828926A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:316)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
	at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:ABEC0F3DC89AD48C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:308)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13045 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_C03129E07D75B94-001/init-core-data-001
   [junit4]   2> 1531590 WARN  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1531591 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1531592 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1531601 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1531637 DEBUG (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 1531638 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1531638 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1531639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1531639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1531644 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1531669 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 1531671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1531678 DEBUG (ScheduledTrigger-3170-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531685 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
   [junit4]   2> 1531686 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1531686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1531687 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1531688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1531688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531688 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1531689 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1531689 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1531692 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1531693 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1531696 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1531696 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1531697 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 1531698 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531698 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1531698 DEBUG (ScheduledTrigger-3175-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531722 DEBUG (ScheduledTrigger-3175-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531742 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531763 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531783 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531803 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531816 DEBUG (simCloudManagerPool-3174-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1531816 DEBUG (simCloudManagerPool-3174-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1531817 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1531818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1531823 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531842 DEBUG (simCloudManagerPool-3174-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1531842 DEBUG (simCloudManagerPool-3174-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1531842 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1532096 DEBUG (simCloudManagerPool-3174-thread-26) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard2: {"core_node18":{
   [junit4]   2>     "core":"testSearchRate_shard2_replica_n18",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testSearchRate"}}
   [junit4]   2> 1532096 DEBUG (simCloudManagerPool-3174-thread-27) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard1: {"core_node1":{
   [junit4]   2>     "core":"testSearchRate_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10079_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testSearchRate"}}
   [junit4]   2> 1532102 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.s.TestLargeCluster Ready after 462 ms
   [junit4]   2> 1532110 DEBUG (simCloudManagerPool-3174-thread-28) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1532111 DEBUG (simCloudManagerPool-3174-thread-28) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1532112 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1532115 WARN  (ScheduledTrigger-3175-thread-2) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532128 DEBUG (simCloudManagerPool-3174-thread-29) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1532129 DEBUG (simCloudManagerPool-3174-thread-29) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1532129 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1532138 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532160 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532182 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532205 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532228 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532235 DEBUG (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6aec54b8ab5e51Td0li4wkf29wzmsxbhziqx06ni",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":30096196150976081,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"testSearchRate_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10079_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node4\":{\n    \"core\":\"testSearchRate_shard1_replica_n4\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10070_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node9\":{\n    \"core\":\"testSearchRate_shard1_replica_n9\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10038_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node5\":{\n    \"core\":\"testSearchRate_shard1_replica_n5\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node3\":{\n    \"core\":\"testSearchRate_shard1_replica_n3\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10093_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node10\":{\n    \"core\":\"testSearchRate_shard1_replica_n10\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10083_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node6\":{\n    \"core\":\"testSearchRate_shard1_replica_n6\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10048_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node8\":{\n    \"core\":\"testSearchRate_shard1_replica_n8\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10064_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"testSearchRate_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10050_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node7\":{\n    \"core\":\"testSearchRate_shard1_replica_n7\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10072_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"testSearchRate":{"shard1":40.0}},
   [junit4]   2>     "hotCollections":{"testSearchRate":40.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10050_solr":4.0,
   [junit4]   2>       "127.0.0.1:10079_solr":4.0,
   [junit4]   2>       "127.0.0.1:10070_solr":4.0,
   [junit4]   2>       "127.0.0.1:10038_solr":4.0,
   [junit4]   2>       "127.0.0.1:10083_solr":4.0,
   [junit4]   2>       "127.0.0.1:10032_solr":4.0,
   [junit4]   2>       "127.0.0.1:10093_solr":4.0,
   [junit4]   2>       "127.0.0.1:10048_solr":4.0,
   [junit4]   2>       "127.0.0.1:10072_solr":4.0,
   [junit4]   2>       "127.0.0.1:10064_solr":4.0},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 1532237 DEBUG (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [search_rate_trigger]
   [junit4]   2> 1532290 DEBUG (simCloudManagerPool-3174-thread-30) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10040_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1532301 DEBUG (AutoscalingActionExecutor-3176-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6aec54b8ab5e51Td0li4wkf29wzmsxbhziqx06ni",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime"

[...truncated too long message...]

file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 84ms :: artifacts dl 6ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 181 minutes 28 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 82 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/82/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds

Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testMixedBounds_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "core":"testMixedBounds_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1529724483343326950",       "replicas":{         "core_node1":{           "core":"testMixedBounds_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495},         "core_node2":{           "core":"testMixedBounds_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1529724483377442800",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247},         "core_node9":{           "core":"testMixedBounds_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1529724483377075950",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248},         "core_node8":{           "core":"testMixedBounds_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248}}}}}

Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1529724483343326950",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377442800",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377075950",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
	at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:69C9F5F280109404]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:582)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1529724483343326950",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377442800",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377075950",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
	... 40 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:817CDD5464EC73]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:48B59F0457D38A8E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:324)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14316 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/init-core-data-001
   [junit4]   2> 3688690 WARN  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 3688690 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3688692 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3688693 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 3688693 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001
   [junit4]   2> 3688694 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3688694 INFO  (Thread-5827) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3688694 INFO  (Thread-5827) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3688696 ERROR (Thread-5827) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3688794 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:53190
   [junit4]   2> 3688796 INFO  (zkConnectionManagerCallback-7590-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688807 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3688808 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c966ddf{/solr,null,AVAILABLE}
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@696914b5{SSL,[ssl, http/1.1]}{127.0.0.1:51482}
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.Server Started @3688973ms
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51482}
   [junit4]   2> 3688810 ERROR (jetty-launcher-7587-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-23T03:27:12.390Z
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3688817 INFO  (zkConnectionManagerCallback-7592-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688818 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3688818 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a239678{/solr,null,AVAILABLE}
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2e12bbf1{SSL,[ssl, http/1.1]}{127.0.0.1:50167}
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.Server Started @3688981ms
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50167}
   [junit4]   2> 3688819 ERROR (jetty-launcher-7587-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-23T03:27:12.398Z
   [junit4]   2> 3688824 INFO  (zkConnectionManagerCallback-7594-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688824 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3688988 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53190/solr
   [junit4]   2> 3688996 INFO  (zkConnectionManagerCallback-7598-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688998 INFO  (zkConnectionManagerCallback-7600-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689177 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53190/solr
   [junit4]   2> 3689181 INFO  (zkConnectionManagerCallback-7606-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689185 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x106faba7c630005, likely client has closed socket
   [junit4]   2> 3689195 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3689196 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51482_solr
   [junit4]   2> 3689197 INFO  (zkConnectionManagerCallback-7608-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689197 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.Overseer Overseer (id=74022122755915780-127.0.0.1:51482_solr-n_0000000000) starting
   [junit4]   2> 3689209 INFO  (zkConnectionManagerCallback-7615-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689209 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3689210 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50167_solr
   [junit4]   2> 3689211 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689213 INFO  (zkCallback-7599-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689213 INFO  (zkCallback-7607-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689216 INFO  (zkCallback-7614-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689219 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3689219 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51482_solr
   [junit4]   2> 3689224 INFO  (zkCallback-7607-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689224 INFO  (zkCallback-7599-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689224 INFO  (zkCallback-7614-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689237 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 3689238 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 3689239 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 3689239 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 3689239 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689249 INFO  (zkConnectionManagerCallback-7621-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689250 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3689253 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:50167_solr, 127.0.0.1:51482_solr]
   [junit4]   2> 3689253 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689260 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689273 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3689273 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 3689283 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689287 DEBUG (ScheduledTrigger-12524-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689295 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689298 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689299 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689300 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001/node1/.
   [junit4]   2> 3689315 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689315 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689317 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001/node2/.
   [junit4]   2> 3689379 INFO  (zkConnectionManagerCallback-7624-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689382 INFO  (zkConnectionManagerCallback-7629-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689383 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3689384 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689439 DEBUG (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 3689440 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 3689440 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 3689441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 3689441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 3689447 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 3689449 DEBUG (ScheduledTrigger-12537-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689470 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689491 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689513 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689533 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689553 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689573 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689593 DEBUG (ScheduledTrigger-12537-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689614 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689634 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689655 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689675 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689693 INFO  (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[634A4A5FCDAB9F5E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMixedBounds
   [junit4]   2> 3689695 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689717 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689738 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689760 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689779 DEBUG (simCloudManagerPool-12536-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMixedBounds_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testMixedBounds_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testMixedBounds_collection"}}
   [junit4]   2> 3689780 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689800 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689803 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689815 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689817 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689817 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689820 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689824 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689824 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689830 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689831 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689831 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689832 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689834 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689834 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689836 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689836 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689837 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689838 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689838 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689839 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689841 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689855 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689857 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":

[...truncated too long message...]

:: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 44ms :: artifacts dl 2ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 176 minutes 29 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 81 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/81/

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup

Error Message:
should be at least one inactive event

Stack Trace:
java.lang.AssertionError: should be at least one inactive event
	at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:ED7B7A4DFF360C39]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic

Error Message:
{} expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: {} expected:<1> but was:<0>
	at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:5BADA72A41A9AD1C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12525 lines...]
   [junit4] Suite: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_F057BA3F9E752B32-001/init-core-data-001
   [junit4]   2> 506119 WARN  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=31 numCloses=31
   [junit4]   2> 506119 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 506121 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 506121 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 506210 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 506323 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasic
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrRrdBackendFactoryTest -Dtests.method=testBasic -Dtests.seed=F057BA3F9E752B32 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL -Dtests.timezone=CAT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.24s J2 | SolrRrdBackendFactoryTest.testBasic <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: {} expected:<1> but was:<0>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:5BADA72A41A9AD1C]:0)
   [junit4]    > 	at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_F057BA3F9E752B32-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3004bc3e), locale=es-CL, timezone=CAT
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=45397472,total=447217664
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialRPTFieldTypeTest, TestManagedResourceStorage, MetricsConfigTest, TestApiFramework, TestMultiWordSynonyms, DocValuesTest, ReplicaListTransformerTest, TestFieldCacheReopen, TestPolicyCloud, AutoAddReplicasPlanActionTest, TestDistributedMissingSort, HttpPartitionTest, DistributedQueueTest, ActionThrottleTest, ClusterStateTest, TestPolicyCloud, CircularListTest, SolrRrdBackendFactoryTest]
   [junit4] Completed [63/821 (1!)] on J2 in 0.58s, 1 test, 1 failure <<< FAILURES!

[...truncated 436 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/init-core-data-001
   [junit4]   2> 1431131 WARN  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 1431132 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1431166 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1431166 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1431167 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/tempDir-001
   [junit4]   2> 1431167 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1431185 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1431185 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1431250 ERROR (Thread-1050) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1431396 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46204
   [junit4]   2> 1431469 INFO  (zkConnectionManagerCallback-1052-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1431492 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1431564 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47a76f97{/solr,null,AVAILABLE}
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5e762bfb{SSL,[ssl, http/1.1]}{127.0.0.1:35194}
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.Server Started @1431710ms
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35194}
   [junit4]   2> 1431646 ERROR (jetty-launcher-1049-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-21T11:39:08.148Z
   [junit4]   2> 1431733 INFO  (zkConnectionManagerCallback-1054-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1431746 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1432502 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46204/solr
   [junit4]   2> 1432538 INFO  (zkConnectionManagerCallback-1058-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1432630 INFO  (zkConnectionManagerCallback-1060-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1433125 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1433126 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35194_solr
   [junit4]   2> 1433126 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.Overseer Overseer (id=72838688041664515-127.0.0.1:35194_solr-n_0000000000) starting
   [junit4]   2> 1433212 INFO  (zkConnectionManagerCallback-1067-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1433233 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46204/solr ready
   [junit4]   2> 1433270 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1433271 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1433272 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1433272 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1433281 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35194_solr
   [junit4]   2> 1433307 INFO  (zkCallback-1059-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1433307 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: []
   [junit4]   2> 1433308 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1433308 DEBUG (ScheduledTrigger-2321-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 0
   [junit4]   2> 1433308 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1433362 INFO  (zkCallback-1066-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1433423 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1433739 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433867 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433867 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433868 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/tempDir-001/node1/.
   [junit4]   2> 1434005 INFO  (zkConnectionManagerCallback-1070-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1434138 INFO  (zkConnectionManagerCallback-1075-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1434139 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1434140 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46204/solr ready
   [junit4]   2> 1434277 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10005_solr
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1434279 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1434279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10005_solr]
   [junit4]   2> 1434280 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1434280 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1434280 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434301 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434372 DEBUG (ScheduledTrigger-2321-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434406 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434443 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434480 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434516 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434554 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434571 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testInactiveShardCleanup
   [junit4]   2> 1434589 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434609 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434772 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434837 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434857 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434894 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434931 DEBUG (simCloudManagerPool-2329-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435004 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435004 DEBUG (simCloudManagerPool-2329-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_1: {"core_node5":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_1_replica0",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435005 DEBUG (simCloudManagerPool-2329-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_0: {"core_node4":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":20480,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":1.9073486328125E-5,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435040 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435113 DEBUG (ScheduledTrigger-2330-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435139 DEBUG (simCloudManagerPool-2329-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435139 DEBUG (simCloudManagerPool-2329-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1435202 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1435223 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1435228 DEBUG (simCloudManagerPool-2329-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435228 DEBUG (simCloudManagerPool-2329-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1435239 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger .scheduled_maintenance firing registered processor for scheduled time 2018-06-22T05:50:50.320Z, now=2018-06-22T05:51:36.334Z
   [junit4]   2> 1435259 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435272 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{"actualEventTime":1529646696334}}
   [junit4]   2> 1435272 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 1435333 DEBUG (simCloudManagerPool-2329-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1435334 INFO  (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984102029588133, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435349 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}
   [junit4]   2> 1435361 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984103400658833, stage=BEFORE_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435363 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction ScheduledMaintenanceTriggerTest_collection1/shard1: tstamp=1529646680111887450, time=1529646702562729550, delta=22
   [junit4]   2> 1435363 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction -- delete inactive ScheduledMaintenanceTriggerTest_collection1 / shard1
   [junit4]   2> 1435434 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984106240219833, stage=AFTER_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard, method=GET, params.action=DELETESHARD, params.collection=ScheduledMaintenanceTriggerTest_collection1, params.shard=shard1}], properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance, properties.inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, properties.AFTER_ACTION=[inactive_shard_plan]}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435434 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard@41ffc1cf], inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, AFTER_ACTION=[inactive_shard_plan], BEFORE_ACTION=[inactive_shard_plan, execute_plan]}
   [junit4]   2> 1435434 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=DELETESHARD&collection=ScheduledMaintenanceTriggerTest_collection1&shard=shard1
   [junit4]   2> 1435435 DEBUG (ScheduledTrigger-2321-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435615 DEBUG (simCloudManagerPool-2329-thread-16) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1435615 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984116100425783, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 13325 ms for event id=2a937977b32099T404d9u5tlnz465c0ra41w4vyp
   [junit4]   2> 1435616 DEBUG (simCloudManagerPool-2329-thread-17) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435616 DEBUG (simCloudManagerPool-2329-thread-17) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1435649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1435686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1435715 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435747 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435751 DEBUG (simCloudManagerPool-2329-thread-18) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435751 DEBUG (simCloudManagerPool-2329-thread-18) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1435763 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testInactiveShardCleanup
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=F057BA3F9E752B32 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=uk-UA -Dtests.timezone=Indian/Mahe -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 1.28s J2 | ScheduledMaintenanceTriggerTest.testInactiveShardCleanup <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: should be at least one inactive event
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:ED7B7A4DFF360C39]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1435783 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435807 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435828 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435909 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435995 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerDefaults
   [junit4]   2> 1435995 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest {
   [junit4]   2>   "cluster-preferences":[
   [junit4]   2>     {
   [junit4]   2>       "minimize":"cores",
   [junit4]   2>       "precision":1},
   [junit4]   2>     {"maximize":"freedisk"}],
   [junit4]   2>   "triggers":{
   [junit4]   2>     ".scheduled_maintenance":{
   [junit4]   2>       "startTime":"NOW",
   [junit4]   2>       "event":"scheduled",
   [junit4]   2>       "every":"+1DAY",
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"inactive_shard_plan",
   [junit4]   2>           "class":"solr.InactiveShardPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true},
   [junit4]   2>     ".auto_add_replicas":{
   [junit4]   2>       "name":".auto_add_replicas",
   [junit4]   2>       "event":"nodeLost",
   [junit4]   2>       "waitFor":120,
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"auto_add_replicas_plan",
   [junit4]   2>           "class":"solr.AutoAddReplicasPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true}},
   [junit4]   2>   "listeners":{
   [junit4]   2>     ".scheduled_maintenance.system":{
   [junit4]   2>       "trigger":".scheduled_maintenance",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]},
   [junit4]   2>     ".auto_add_replicas.system":{
   [junit4]   2>       "trigger":".auto_add_replicas",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]}},
   [junit4]   2>   "properties":{}}
   [junit4]   2> 1436007 DEBUG (ScheduledTrigger-2330-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1436008 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerDefaults
   [junit4]   2> 1436008 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1436009 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1436019 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1436019 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1436137 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e762bfb{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1436141 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=554313082
   [junit4]   2> 1436178 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1436178 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e084578: rootName = solr_35194, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@15b3e449
   [junit4]   2> 1436195 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1436195 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e8995e9: rootName = solr_35194, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6328312c
   [junit4]   2> 1436251 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1436251 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d421042: rootName = solr_35194, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2946d62e
   [junit4]   2> 1436289 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35194_solr
   [junit4]   2> 1436291 INFO  (zkCallback-1059-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1436291 INFO  (zkCallback-1066-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1436291 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1436320 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1436320 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72838688041664515-127.0.0.1:35194_solr-n_0000000000) closing
   [junit4]   2> 1436320 INFO  (OverseerStateUpdate-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35194_solr
   [junit4]   2> 1436361 WARN  (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1436362 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1436362 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1436364 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@47a76f97{/solr,null,UNAVAILABLE}
   [junit4]   2> 1436364 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1436413 ERROR (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1436414 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46204 46204
   [junit4]   2> 1436521 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46204 46204
   [junit4]   2> 1436568 WARN  (Thread-1050) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/live_nodes
   [junit4]   2> 	3	/solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1819, maxMBSortInHeap=7.400451758482797, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3a1d7183), locale=uk-UA, timezone=Indian/Mahe
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=248162552,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialRPTFieldTypeTest, TestManagedResourceStorage, MetricsConfigTest, TestApiFramework, TestMultiWordSynonyms, DocValuesTest, ReplicaListTransformerTest, TestFieldCacheReopen, TestPolicyCloud, AutoAddReplicasPlanActionTest, TestDistributedMissingSort, HttpPartitionTest, DistributedQueueTest, ActionThrottleTest, ClusterStateTest, TestPolicyCloud, CircularListTest, SolrRrdBackendFactoryTest, TestCloudJSONFacetJoinDomain, CoreAdminRequestStatusTest, TestRangeQuery, TestSafeXMLParsing, DistributedSuggestComponentTest, AddSchemaFieldsUpdateProcessorFactoryTest, RegexBoostProcessorTest, UtilsToolTest, ResourceLoaderTest, SuggestComponentContextFilterQueryTest, TestRandomDVFaceting, OpenExchangeRatesOrgProviderTest, TestCorePropertiesReload, ConnectionManagerTest, DeleteReplicaTest, TestSkipOverseerOperations, UniqFieldsUpdateProcessorFactoryTest, AnalyticsMergeStrategyTest, MetricsHandlerTest, BaseCdcrDistributedZkTest, SpatialFilterTest, SearchRateTriggerIntegrationTest, BasicDistributedZk2Test, TestMultiValuedNumericRangeQuery, FileBasedSpellCheckerTest, ShowFileRequestHandlerTest, TestJmxIntegration, ScheduledMaintenanceTriggerTest]
   [junit4] Completed [197/821 (2!)] on J2 in 5.59s, 2 tests, 1 failure <<< FAILURES!

[...truncated 52480 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
     [java] Processed 2243 links (1792 relative) to 3127 anchors in 247 files
     [echo] Validated Links & Anchors via: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-ref-guide/bare-bones-html/

-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec] 
     [exec] Crawl/parse...
     [exec] 
     [exec] Verify...
     [echo] Checking for malformed docs...

jar-checksums:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

common.resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 28ms :: artifacts dl 4ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 211 minutes 32 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)