You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/02/16 05:49:46 UTC

[JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk1.8.0_201) - Build # 2153 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/2153/
Java: 64bit/jdk1.8.0_201 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestSimScenario.testIndexing

Error Message:
Timed out waiting for trigger indexSizeTrigger to fire after simulated 60s (real 1200ms).

Stack Trace:
java.io.IOException: Timed out waiting for trigger indexSizeTrigger to fire after simulated 60s (real 1200ms).
	at __randomizedtesting.SeedInfo.seed([D2A436B5FFEF25BA:367E79BCD99A6631]:0)
	at org.apache.solr.cloud.autoscaling.sim.SimScenario$SimWaitListener.wait(SimScenario.java:693)
	at org.apache.solr.cloud.autoscaling.sim.SimScenario$WaitEvent.execute(SimScenario.java:758)
	at org.apache.solr.cloud.autoscaling.sim.SimScenario.run(SimScenario.java:1096)
	at org.apache.solr.cloud.autoscaling.sim.TestSimScenario.testIndexing(TestSimScenario.java:137)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13692 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimScenario
   [junit4]   2> 104419 INFO  (SUITE-TestSimScenario-seed#[D2A436B5FFEF25BA]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 104419 INFO  (SUITE-TestSimScenario-seed#[D2A436B5FFEF25BA]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 104434 INFO  (SUITE-TestSimScenario-seed#[D2A436B5FFEF25BA]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J3/temp/solr.cloud.autoscaling.sim.TestSimScenario_D2A436B5FFEF25BA-001/data-dir-21-001
   [junit4]   2> 104435 INFO  (SUITE-TestSimScenario-seed#[D2A436B5FFEF25BA]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 104436 INFO  (SUITE-TestSimScenario-seed#[D2A436B5FFEF25BA]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 104439 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testSuggestions
   [junit4]   2> 104471 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 1.	CreateCluster	numNodes=2
   [junit4]   2> 104472 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		CreateCluster	numNodes=2
   [junit4]   2> 104526 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 104526 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
   [junit4]   2> 104526 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 104527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 104527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 104527 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 104528 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 104528 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 104528 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 2.	RunSolrRequest	path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 104528 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 104530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 104530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 104530 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 104531 DEBUG (ScheduledTrigger-746-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104531 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCollection, currentVersion=1
   [junit4]   2> 104531 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
   [junit4]   2> 104532 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 104543 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
   [junit4]   2> 104549 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 104555 DEBUG (ScheduledTrigger-746-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104577 DEBUG (ScheduledTrigger-746-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104580 DEBUG (simCloudManagerPool-745-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
   [junit4]   2> 104581 DEBUG (simCloudManagerPool-745-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 104581 DEBUG (simCloudManagerPool-745-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / shard1)
   [junit4]   2> 104582 DEBUG (simCloudManagerPool-745-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / shard1 (currentVersion=4): {"core_node1":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 104583 DEBUG (simCloudManagerPool-745-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCollection, currentVersion=4
   [junit4]   2> 104584 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 3.	WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 104584 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 104594 DEBUG (simCloudManagerPool-745-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
   [junit4]   2> 104595 DEBUG (simCloudManagerPool-745-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 104595 DEBUG (simCloudManagerPool-745-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / shard2)
   [junit4]   2> 104595 DEBUG (simCloudManagerPool-745-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / shard2 (currentVersion=5): {"core_node3":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n3",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 104597 DEBUG (ScheduledTrigger-746-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104606 DEBUG (simCloudManagerPool-745-thread-3) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
   [junit4]   2> 104606 DEBUG (simCloudManagerPool-745-thread-3) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 104617 DEBUG (ScheduledTrigger-746-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104618 DEBUG (simCloudManagerPool-745-thread-5) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
   [junit4]   2> 104619 DEBUG (simCloudManagerPool-745-thread-5) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 104619 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
   [junit4]   2> 104619 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
   [junit4]   2> 104619 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 4.	CtxSet	key=myNode&value=${_random_node_}
   [junit4]   2> 104619 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		CtxSet	key=myNode&value=127.0.0.1:10000_solr
   [junit4]   2> 104620 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 5.	RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 104620 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10000_solr
   [junit4]   2> 104623 INFO  (simCloudManagerPool-745-thread-6) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 104623 INFO  (simCloudManagerPool-745-thread-6) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 104638 DEBUG (ScheduledTrigger-746-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104644 DEBUG (simCloudManagerPool-745-thread-6) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
   [junit4]   2> 104644 DEBUG (simCloudManagerPool-745-thread-6) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
   [junit4]   2> 104644 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 6.	RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 104644 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10000_solr
   [junit4]   2> 104645 DEBUG (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
   [junit4]   2> 104645 DEBUG (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
   [junit4]   2> 104646 INFO  (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 104646 INFO  (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 104656 DEBUG (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
   [junit4]   2> 104656 DEBUG (simCloudManagerPool-745-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
   [junit4]   2> 104656 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 7.	LoopOp	iterations=${iterative}
   [junit4]   2> 104656 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		LoopOp	iterations=0
   [junit4]   2> 104657 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 8.	LoopOp	iterations=${justCalc}
   [junit4]   2> 104657 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		LoopOp	iterations=1
   [junit4]   2> 104657 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 1:
   [junit4]   2> 104657 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 104658 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
   [junit4]   2> 104659 DEBUG (ScheduledTrigger-746-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104659 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
   [junit4]   2> 104679 DEBUG (ScheduledTrigger-746-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104685 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 104685 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - SaveSnapshot	path=/home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J3/temp/solr.cloud.autoscaling.sim.TestSimScenario_D2A436B5FFEF25BA-001/tempDir-001/snapshot/0
   [junit4]   2> 104693 WARN  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimUtils V2 action SET_CLUSTER_PROPERTY_OBJ has no equivalent V1 action
   [junit4]   2> 104693 WARN  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimUtils V2 action GET_NODES has no equivalent V1 action
   [junit4]   2> 104699 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SnapshotDistribStateManager - copying 22 resources from SimDistribStateManager
   [junit4]   2> 104699 DEBUG (ScheduledTrigger-746-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104720 DEBUG (ScheduledTrigger-746-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104740 DEBUG (ScheduledTrigger-746-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104747 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 9.	Dump	redact=true
   [junit4]   2> 104747 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		Dump	redact=true
   [junit4]   2> 104748 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SnapshotDistribStateManager - copying 22 resources from SimDistribStateManager
   [junit4]   2> 104757 INFO  (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 104758 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 104759 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 104759 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 104759 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 104759 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 104759 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 104759 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 1.	CreateCluster	numNodes=2
   [junit4]   2> 104759 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		CreateCluster	numNodes=2
   [junit4]   2> 104773 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10002_solr
   [junit4]   2> 104773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 104773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 104774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 104774 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 104784 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10002_solr]
   [junit4]   2> 104788 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
   [junit4]   2> 104788 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
   [junit4]   2> 104788 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 104788 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 2.	RunSolrRequest	path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 104788 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
   [junit4]   2> 104788 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 104788 DEBUG (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 104789 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testCollection, currentVersion=1
   [junit4]   2> 104789 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
   [junit4]   2> 104802 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
   [junit4]   2> 104802 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 1
   [junit4]   2> 104812 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
   [junit4]   2> 104813 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
   [junit4]   2> 104823 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104825 DEBUG (simCloudManagerPool-751-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
   [junit4]   2> 104825 DEBUG (simCloudManagerPool-751-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
   [junit4]   2> 104825 DEBUG (simCloudManagerPool-751-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / shard1)
   [junit4]   2> 104825 DEBUG (simCloudManagerPool-751-thread-2) [     ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / shard1 (currentVersion=4): {"core_node1":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n1",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 104836 DEBUG (simCloudManagerPool-751-thread-3) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
   [junit4]   2> 104836 DEBUG (simCloudManagerPool-751-thread-3) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
   [junit4]   2> 104848 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104848 DEBUG (simCloudManagerPool-751-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
   [junit4]   2> 104850 DEBUG (simCloudManagerPool-751-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
   [junit4]   2> 104850 DEBUG (simCloudManagerPool-751-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider Running leader election (testCollection / shard2)
   [junit4]   2> 104850 DEBUG (simCloudManagerPool-751-thread-4) [     ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testCollection / shard2 (currentVersion=6): {"core_node3":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n3",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 104851 DEBUG (simCloudManagerPool-751-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testCollection, currentVersion=6
   [junit4]   2> 104851 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 3.	WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 104851 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 104852 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
   [junit4]   2> 104853 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
   [junit4]   2> 104867 DEBUG (simCloudManagerPool-751-thread-5) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
   [junit4]   2> 104867 DEBUG (simCloudManagerPool-751-thread-5) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
   [junit4]   2> 104867 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
   [junit4]   2> 104868 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
   [junit4]   2> 104868 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 4.	CtxSet	key=myNode&value=${_random_node_}
   [junit4]   2> 104868 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		CtxSet	key=myNode&value=127.0.0.1:10003_solr
   [junit4]   2> 104868 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 5.	RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 104868 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 104868 INFO  (simCloudManagerPool-751-thread-6) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 104868 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104868 INFO  (simCloudManagerPool-751-thread-6) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 104879 DEBUG (simCloudManagerPool-751-thread-6) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
   [junit4]   2> 104879 DEBUG (simCloudManagerPool-751-thread-6) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
   [junit4]   2> 104879 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 6.	RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
   [junit4]   2> 104879 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 104880 DEBUG (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
   [junit4]   2> 104880 DEBUG (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
   [junit4]   2> 104881 INFO  (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 104881 INFO  (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 104888 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104891 DEBUG (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
   [junit4]   2> 104891 DEBUG (simCloudManagerPool-751-thread-7) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
   [junit4]   2> 104891 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 7.	LoopOp	iterations=${iterative}
   [junit4]   2> 104891 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		LoopOp	iterations=10
   [junit4]   2> 104891 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 1:
   [junit4]   2> 104891 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 104891 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
   [junit4]   2> 104892 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
   [junit4]   2> 104893 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 104893 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 104911 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104932 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104953 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104973 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104994 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 104995 DEBUG (simCloudManagerPool-751-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node10":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n9",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 105014 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105035 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105055 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105076 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105096 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105109 DEBUG (simCloudManagerPool-751-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
   [junit4]   2> 105109 DEBUG (simCloudManagerPool-751-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
   [junit4]   2> 105109 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105110 DEBUG (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
   [junit4]   2> 105110 DEBUG (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
   [junit4]   2> 105110 INFO  (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105110 INFO  (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105116 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105121 DEBUG (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
   [junit4]   2> 105122 DEBUG (simCloudManagerPool-751-thread-9) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
   [junit4]   2> 105122 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105125 DEBUG (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
   [junit4]   2> 105125 DEBUG (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
   [junit4]   2> 105126 INFO  (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105126 INFO  (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105136 DEBUG (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
   [junit4]   2> 105136 DEBUG (simCloudManagerPool-751-thread-10) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
   [junit4]   2> 105136 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 2:
   [junit4]   2> 105136 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 105136 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
   [junit4]   2> 105137 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105137 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
   [junit4]   2> 105138 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 105138 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 105157 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105177 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105198 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105218 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105239 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105239 DEBUG (simCloudManagerPool-751-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node16":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n15",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 105259 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105279 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105300 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105320 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105341 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105350 DEBUG (simCloudManagerPool-751-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
   [junit4]   2> 105350 DEBUG (simCloudManagerPool-751-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
   [junit4]   2> 105351 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105354 DEBUG (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
   [junit4]   2> 105355 DEBUG (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
   [junit4]   2> 105355 INFO  (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105355 INFO  (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105361 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105365 DEBUG (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
   [junit4]   2> 105366 DEBUG (simCloudManagerPool-751-thread-12) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
   [junit4]   2> 105366 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105367 DEBUG (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
   [junit4]   2> 105367 DEBUG (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
   [junit4]   2> 105367 INFO  (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105367 INFO  (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105378 DEBUG (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
   [junit4]   2> 105379 DEBUG (simCloudManagerPool-751-thread-13) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
   [junit4]   2> 105379 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 3:
   [junit4]   2> 105379 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 105379 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
   [junit4]   2> 105382 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105383 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
   [junit4]   2> 105384 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 105384 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 105403 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105424 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105444 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105465 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105485 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105488 DEBUG (simCloudManagerPool-751-thread-14) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node22":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n21",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 105506 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105526 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105546 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105567 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105587 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105598 DEBUG (simCloudManagerPool-751-thread-14) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=25
   [junit4]   2> 105598 DEBUG (simCloudManagerPool-751-thread-14) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 25
   [junit4]   2> 105598 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105600 DEBUG (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=26
   [junit4]   2> 105600 DEBUG (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 26
   [junit4]   2> 105601 INFO  (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105601 INFO  (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105608 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105611 DEBUG (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=27
   [junit4]   2> 105623 DEBUG (simCloudManagerPool-751-thread-15) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 27
   [junit4]   2> 105623 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105624 DEBUG (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=28
   [junit4]   2> 105625 DEBUG (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 28
   [junit4]   2> 105625 INFO  (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105625 INFO  (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105628 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105635 DEBUG (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=29
   [junit4]   2> 105636 DEBUG (simCloudManagerPool-751-thread-16) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 29
   [junit4]   2> 105636 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 4:
   [junit4]   2> 105636 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 105636 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=30
   [junit4]   2> 105637 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 30
   [junit4]   2> 105638 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 105638 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 105648 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105669 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105689 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105710 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105730 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105741 DEBUG (simCloudManagerPool-751-thread-17) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node28":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n27",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 105751 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105771 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105791 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105811 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105832 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105851 DEBUG (simCloudManagerPool-751-thread-17) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=31
   [junit4]   2> 105852 DEBUG (simCloudManagerPool-751-thread-17) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 31
   [junit4]   2> 105852 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105853 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105856 DEBUG (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=32
   [junit4]   2> 105857 DEBUG (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 32
   [junit4]   2> 105858 INFO  (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105858 INFO  (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105868 DEBUG (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=33
   [junit4]   2> 105868 DEBUG (simCloudManagerPool-751-thread-18) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 33
   [junit4]   2> 105869 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 105869 DEBUG (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=34
   [junit4]   2> 105869 DEBUG (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 34
   [junit4]   2> 105869 INFO  (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 105869 INFO  (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 105873 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105880 DEBUG (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=35
   [junit4]   2> 105881 DEBUG (simCloudManagerPool-751-thread-19) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 35
   [junit4]   2> 105881 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 5:
   [junit4]   2> 105881 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 105881 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=36
   [junit4]   2> 105882 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 36
   [junit4]   2> 105883 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 105883 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 105893 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105914 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105935 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105955 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105976 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 105985 DEBUG (simCloudManagerPool-751-thread-20) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node34":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n33",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 105996 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106017 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106037 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106057 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106078 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106098 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106100 DEBUG (simCloudManagerPool-751-thread-20) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=37
   [junit4]   2> 106100 DEBUG (simCloudManagerPool-751-thread-20) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 37
   [junit4]   2> 106100 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106104 DEBUG (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=38
   [junit4]   2> 106105 DEBUG (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 38
   [junit4]   2> 106106 INFO  (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106106 INFO  (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106119 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106120 DEBUG (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=39
   [junit4]   2> 106121 DEBUG (simCloudManagerPool-751-thread-21) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 39
   [junit4]   2> 106121 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106125 DEBUG (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=40
   [junit4]   2> 106126 DEBUG (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 40
   [junit4]   2> 106126 INFO  (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106127 INFO  (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106137 DEBUG (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=41
   [junit4]   2> 106137 DEBUG (simCloudManagerPool-751-thread-22) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 41
   [junit4]   2> 106137 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 6:
   [junit4]   2> 106137 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 106137 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=42
   [junit4]   2> 106138 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 42
   [junit4]   2> 106138 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 106138 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 106139 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106159 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106179 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106204 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106224 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106240 DEBUG (simCloudManagerPool-751-thread-23) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node40":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n39",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 106244 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106266 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106286 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106307 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106327 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106347 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106351 DEBUG (simCloudManagerPool-751-thread-23) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=43
   [junit4]   2> 106351 DEBUG (simCloudManagerPool-751-thread-23) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 43
   [junit4]   2> 106351 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106359 DEBUG (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=44
   [junit4]   2> 106361 DEBUG (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 44
   [junit4]   2> 106361 INFO  (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106362 INFO  (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106367 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106372 DEBUG (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=45
   [junit4]   2> 106372 DEBUG (simCloudManagerPool-751-thread-24) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 45
   [junit4]   2> 106373 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106373 DEBUG (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=46
   [junit4]   2> 106373 DEBUG (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 46
   [junit4]   2> 106374 INFO  (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106374 INFO  (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106386 DEBUG (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=47
   [junit4]   2> 106387 DEBUG (simCloudManagerPool-751-thread-25) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 47
   [junit4]   2> 106387 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 7:
   [junit4]   2> 106387 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 106387 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=48
   [junit4]   2> 106388 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 48
   [junit4]   2> 106388 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106389 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 106389 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 106409 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106429 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106449 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106471 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106491 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106494 DEBUG (simCloudManagerPool-751-thread-26) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node46":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n45",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 106512 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106532 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106553 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106575 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106596 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106605 DEBUG (simCloudManagerPool-751-thread-26) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=49
   [junit4]   2> 106605 DEBUG (simCloudManagerPool-751-thread-26) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 49
   [junit4]   2> 106605 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106606 DEBUG (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=50
   [junit4]   2> 106606 DEBUG (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 50
   [junit4]   2> 106607 INFO  (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106607 INFO  (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106617 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106617 DEBUG (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=51
   [junit4]   2> 106618 DEBUG (simCloudManagerPool-751-thread-27) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 51
   [junit4]   2> 106618 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106630 DEBUG (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=52
   [junit4]   2> 106630 DEBUG (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 52
   [junit4]   2> 106631 INFO  (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106631 INFO  (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106637 DEBUG (ScheduledTrigger-752-thread-2) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106641 DEBUG (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=53
   [junit4]   2> 106642 DEBUG (simCloudManagerPool-751-thread-28) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 53
   [junit4]   2> 106642 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 8:
   [junit4]   2> 106642 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 106642 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=54
   [junit4]   2> 106642 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 54
   [junit4]   2> 106643 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 106643 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 106660 DEBUG (ScheduledTrigger-752-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106680 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106701 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106721 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106741 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106747 DEBUG (simCloudManagerPool-751-thread-29) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node52":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n51",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "path":"/admin/collections",
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "action":"ADDREPLICA",
   [junit4]   2>     "createNodeSet":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6}}
   [junit4]   2> 106761 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106782 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106802 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106824 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106844 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106860 DEBUG (simCloudManagerPool-751-thread-29) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=55
   [junit4]   2> 106861 DEBUG (simCloudManagerPool-751-thread-29) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 55
   [junit4]   2> 106864 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106865 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106872 DEBUG (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=56
   [junit4]   2> 106872 DEBUG (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 56
   [junit4]   2> 106873 INFO  (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106873 INFO  (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106883 DEBUG (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=57
   [junit4]   2> 106884 DEBUG (simCloudManagerPool-751-thread-30) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 57
   [junit4]   2> 106884 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - RunSolrRequest	path=/admin/collections&action=ADDREPLICA&collection=testCollection&shard=shard1&node=127.0.0.1:10003_solr
   [junit4]   2> 106884 DEBUG (ScheduledTrigger-752-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106885 DEBUG (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=58
   [junit4]   2> 106892 DEBUG (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 58
   [junit4]   2> 106893 INFO  (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10003_solr for creating new replica of shard shard1 for collection testCollection
   [junit4]   2> 106893 INFO  (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 106904 DEBUG (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=59
   [junit4]   2> 106904 DEBUG (simCloudManagerPool-751-thread-31) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 59
   [junit4]   2> 106904 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario    * iter 9:
   [junit4]   2> 106904 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - CalculateSuggestions	
   [junit4]   2> 106904 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=60
   [junit4]   2> 106905 DEBUG (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 60
   [junit4]   2> 106905 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2 and last live nodes: 2
   [junit4]   2> 106905 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario         - 1 suggestions
   [junit4]   2> 106905 INFO  (TEST-TestSimScenario.testSuggestions-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario      - ApplySuggestions	
   [junit4]   2> 106925 DEBUG (ScheduledTrigger-752-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running No

[...truncated too long message...]

solr",
   [junit4]   2>   "params.inPlaceMove":"true",
   [junit4]   2>   "params.replica":"core_node2"}, {
   [junit4]   2>   "class":"org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica",
   [junit4]   2>   "method":"GET",
   [junit4]   2>   "params.action":"MOVEREPLICA",
   [junit4]   2>   "params.collection":"testCollection",
   [junit4]   2>   "params.targetNode":"127.0.0.1:10108_solr",
   [junit4]   2>   "params.inPlaceMove":"true",
   [junit4]   2>   "params.replica":"core_node3"}], AFTER_ACTION=[auto_add_replicas_plan], BEFORE_ACTION=[auto_add_replicas_plan, execute_plan]}
   [junit4]   2> 111267 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10108_solr&inPlaceMove=true&replica=core_node2
   [junit4]   2> 111368 DEBUG (simCloudManagerPool-775-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node6":{
   [junit4]   2>     "core":"testCollection_shard1_replica_n5",
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10108_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "state":"down",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 111479 DEBUG (simCloudManagerPool-775-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
   [junit4]   2> 111479 DEBUG (simCloudManagerPool-775-thread-8) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
   [junit4]   2> 111487 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testCollection&targetNode=127.0.0.1:10108_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 111488 DEBUG (simCloudManagerPool-775-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
   [junit4]   2> 111488 DEBUG (simCloudManagerPool-775-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
   [junit4]   2> 111588 DEBUG (simCloudManagerPool-775-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node8":{
   [junit4]   2>     "core":"testCollection_shard2_replica_n7",
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testCollection",
   [junit4]   2>     "node_name":"127.0.0.1:10108_solr",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "state":"down",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0}}
   [junit4]   2> 111699 DEBUG (simCloudManagerPool-775-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
   [junit4]   2> 111699 DEBUG (simCloudManagerPool-775-thread-11) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"1b8bfc00941d4Tela9bk752drbyt45d4w9zm7bv",
   [junit4]   2>   "source":".auto_add_replicas",
   [junit4]   2>   "eventTime":484608676807124,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[484608676807124],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":484618825515124,
   [junit4]   2>     "nodeNames":["127.0.0.1:10109_solr"]}}
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 111704 DEBUG (AutoscalingActionExecutor-777-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing took 22823 ms for event id=1b8bfc00941d4Tela9bk752drbyt45d4w9zm7bv
   [junit4]   2> 111704 INFO  (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 8.	WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 111704 INFO  (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		WaitCollection	collection=testCollection&shards=2&replicas=2
   [junit4]   2> 111704 INFO  (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 9.	SaveSnapshot	path=${snapshotPath}
   [junit4]   2> 111704 INFO  (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SimScenario 		SaveSnapshot	path=/home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J3/temp/solr.cloud.autoscaling.sim.TestSimScenario_D2A436B5FFEF25BA-001/tempDir-002/snapshot
   [junit4]   2> 111705 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SnapshotDistribStateManager - copying 23 resources from SimDistribStateManager
   [junit4]   2> 111708 INFO  (Simulated OverseerAutoScalingTriggerThread) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 111708 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 111711 DEBUG (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.c.a.s.SnapshotDistribStateManager - loaded snapshot of 23 resources
   [junit4]   2> 111712 INFO  (TEST-TestSimScenario.testAutoAddReplicas-seed#[D2A436B5FFEF25BA]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testAutoAddReplicas
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-8.x-Linux/solr/build/solr-core/test/J3/temp/solr.cloud.autoscaling.sim.TestSimScenario_D2A436B5FFEF25BA-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene84): {}, docValues:{}, maxPointsInLeafNode=565, maxMBSortInHeap=5.2610537185226365, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6a6e2110), locale=he, timezone=Africa/Banjul
   [junit4]   2> NOTE: Linux 5.3.0-28-generic amd64/Oracle Corporation 1.8.0_201 (64-bit)/cpus=16,threads=1,free=260399592,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [ExitableDirectoryReaderTest, DistributedQueryComponentOptimizationTest, NumericFieldsTest, TestPerFieldSimilarityWithDefaultOverride, ExecutePlanActionTest, AtomicUpdatesTest, TestCoreBackup, TestDynamicLoadingUrl, TestSmileRequest, MoveReplicaHDFSTest, CursorPagingTest, TestLegacyNumericUtils, Tagger2Test, UUIDUpdateProcessorFallbackTest, TestSolrJ, TestFacetMethods, TestLogWatcher, TestExtendedDismaxParser, TermVectorComponentDistributedTest, TestConfigsApi, TestSimScenario]
   [junit4] Completed [113/901 (1!)] on J3 in 7.39s, 4 tests, 1 error <<< FAILURES!

[...truncated 55229 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/2153/consoleText

[repro] Revision: 8fcf500d30bc09a4375174373292685c5c57b442

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC"
[repro] Repro line:  ant test  -Dtestcase=TestSimScenario -Dtests.method=testIndexing -Dtests.seed=D2A436B5FFEF25BA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=he -Dtests.timezone=Africa/Banjul -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestSimScenario
[repro] ant compile-test

[...truncated 3436 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSimScenario" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=D2A436B5FFEF25BA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=he -Dtests.timezone=Africa/Banjul -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[...truncated 91 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestSimScenario
[repro] Exiting with code 0

[...truncated 43 lines...]

[JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-13.0.2) - Build # 2154 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Error processing tokens: Error while parsing action 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space