You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/10/14 20:17:39 UTC
[JENKINS] Lucene-Solr-Tests-7.x - Build # 950 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/950/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction.testNodeWithMultipleReplicasLost
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([B64EC5761226FE96]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([B64EC5761226FE96]:0)
Build Log:
[...truncated 15534 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestSimComputePlanAction_B64EC5761226FE96-001/init-core-data-001
[junit4] 2> 4034697 INFO (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 4034711 INFO (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 4034711 INFO (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 4034731 DEBUG (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10006_solr
[junit4] 2> 4034731 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 4034731 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 4034731 DEBUG (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=0
[junit4] 2> 4034732 DEBUG (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 0
[junit4] 2> 4034732 INFO (SUITE-TestSimComputePlanAction-seed#[B64EC5761226FE96]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 4034732 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 4034732 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 4034751 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10006_solr]
[junit4] 2> 4034776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4034776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 4034776 DEBUG (ScheduledTrigger-8433-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034797 DEBUG (ScheduledTrigger-8433-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034801 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
[junit4] 2> 4034802 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 4034802 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 4034802 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 4034829 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 4034829 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 4034829 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 4034829 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10006_solr]
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4034832 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 4034833 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034855 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034876 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034896 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034929 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034949 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034970 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4034990 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035019 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035040 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035041 DEBUG (simCloudManagerPool-8437-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=1
[junit4] 2> 4035041 DEBUG (simCloudManagerPool-8437-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 1
[junit4] 2> 4035043 DEBUG (simCloudManagerPool-8437-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4035043 DEBUG (simCloudManagerPool-8437-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 4035063 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 4035067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4035067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 4035084 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035084 DEBUG (simCloudManagerPool-8437-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4035084 DEBUG (simCloudManagerPool-8437-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 4035088 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 4035088 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4035088 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 4035104 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035124 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035161 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035181 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035214 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035215 DEBUG (simCloudManagerPool-8437-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4035216 DEBUG (simCloudManagerPool-8437-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 4035216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 4035216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10006_solr]
[junit4] 2> 4035216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction$AssertingTriggerAction}], enabled=true}
[junit4] 2> 4035216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 4035216 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035216 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035217 DEBUG (simCloudManagerPool-8437-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4035218 DEBUG (simCloudManagerPool-8437-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 4035219 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testNodeAdded, currentVersion=2
[junit4] 2> 4035219 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=2
[junit4] 2> 4035219 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 2
[junit4] 2> 4035219 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
[junit4] 2> 4035230 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 4035230 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 4035246 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035246 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035246 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035251 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=3
[junit4] 2> 4035254 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 3
[junit4] 2> 4035254 WARN (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 4035254 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 4035266 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 4035266 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035271 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035271 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035284 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 21940700050 sessionWrapper.createTime 21940699166983320, this.sessionWrapper.createTime 21940699166983320
[junit4] 2> 4035284 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 4035287 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035303 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035303 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035311 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035324 DEBUG (simCloudManagerPool-8437-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testNodeAdded, currentVersion=4
[junit4] 2> 4035327 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035327 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035331 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035332 DEBUG (simCloudManagerPool-8437-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=4
[junit4] 2> 4035333 DEBUG (simCloudManagerPool-8437-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 4
[junit4] 2> 4035348 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035348 DEBUG (ScheduledTrigger-8438-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035352 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035376 DEBUG (simCloudManagerPool-8437-thread-10) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeAdded / shard1 (currentVersion=5): {"core_node1":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 4035389 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035389 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035389 DEBUG (simCloudManagerPool-8437-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=5
[junit4] 2> 4035389 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035389 DEBUG (simCloudManagerPool-8437-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 5
[junit4] 2> 4035409 DEBUG (simCloudManagerPool-8437-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=6
[junit4] 2> 4035409 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035409 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035409 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035415 DEBUG (simCloudManagerPool-8437-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 6
[junit4] 2> 4035445 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035445 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035445 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035445 DEBUG (simCloudManagerPool-8437-thread-9) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=7
[junit4] 2> 4035528 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035528 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035528 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035528 DEBUG (simCloudManagerPool-8437-thread-9) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 7
[junit4] 2> 4035528 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=8
[junit4] 2> 4035569 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035569 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4035569 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4035569 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 8
[junit4] 2> 4035581 DEBUG (simCloudManagerPool-8437-thread-11) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4035581 DEBUG (simCloudManagerPool-8437-thread-11) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 4035581 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 4035582 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 4035591 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035591 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 4035591 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10007_solr at time 21940715404068570
[junit4] 2> 4035591 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 4035611 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4035611 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 4035611 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 4035612 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10007_solr] added at times [21940715404068570], now=21940716424039370
[junit4] 2> 4035612 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4df2f6e3c706daT4vacb6hprqr29sjq30ifo2evn",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":21940715404068570,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[21940715404068570],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "nodeNames":["127.0.0.1:10007_solr"]}}
[junit4] 2> 4035618 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_added_trigger]
[junit4] 2> 4035693 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, currentVersion=9
[junit4] 2> 4035693 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=9
[junit4] 2> 4035693 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 9
[junit4] 2> 4035693 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
[junit4] 2> 4035704 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=10
[junit4] 2> 4035704 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 10
[junit4] 2> 4035704 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 4035705 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 4035706 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 21940721150 sessionWrapper.createTime 21940721103484420, this.sessionWrapper.createTime 21940721103484420
[junit4] 2> 4035706 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 4035717 DEBUG (simCloudManagerPool-8437-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=11
[junit4] 2> 4035717 DEBUG (simCloudManagerPool-8437-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 11
[junit4] 2> 4035720 DEBUG (simCloudManagerPool-8437-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 (currentVersion=12): {"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 4035731 DEBUG (simCloudManagerPool-8437-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=12
[junit4] 2> 4035732 DEBUG (simCloudManagerPool-8437-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 12
[junit4] 2> 4035732 DEBUG (simCloudManagerPool-8437-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, currentVersion=13
[junit4] 2> 4035742 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10007_solr
[junit4] 2> 4035743 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4df2f6e3c706daT4vacb6hprqr29sjq30ifo2evn",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":21940715404068570,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[21940715404068570],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":21940716744154270,
[junit4] 2> "nodeNames":["127.0.0.1:10007_solr"]}}
[junit4] 2> 4035743 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"4df2f6e3c706daT4vacb6hprqr29sjq30ifo2evn",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":21940715404068570,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[21940715404068570],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "_enqueue_time_":21940716744154270,
[junit4] 2> "nodeNames":["127.0.0.1:10007_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 4035743 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 4035744 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 4035744 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node1
[junit4] 2> 4035745 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 21940723075 sessionWrapper.createTime 21940723027154070, this.sessionWrapper.createTime 21940723027154070
[junit4] 2> 4035745 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 4035759 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction Node values: {
[junit4] 2> "127.0.0.1:10006_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":4,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1020,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10006_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10006,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0,
[junit4] 2> "totaldisk":10240},
[junit4] 2> "127.0.0.1:10007_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1022,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10007_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10007,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0,
[junit4] 2> "totaldisk":10240}}
[junit4] 2> 4035759 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction Live nodes: [127.0.0.1:10006_solr, 127.0.0.1:10007_solr], collection state: DocCollection(testNodeAdded//clusterstate.json/12)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"-1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"4",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 4035759 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 4035759 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10006_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":4,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1020,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10006_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10006,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0,
[junit4] 2> "totaldisk":10240},
[junit4] 2> "127.0.0.1:10007_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1022,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10007_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10007,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0,
[junit4] 2> "totaldisk":10240}}
[junit4] 2> 4035759 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction * Live nodes: [127.0.0.1:10006_solr, 127.0.0.1:10007_solr]
[junit4] 2> 4035760 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/12)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":2,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":11240,
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.0468065738677979E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":2},
[junit4] 2> "core_node2":{
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":2,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":11240,
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.0468065738677979E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":2}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 4035760 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction * Collection testNodeAdded state: DocCollection(testNodeAdded//clusterstate.json/12)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"-1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"4",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 4035760 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAdded
[junit4] 2> 4035761 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=13
[junit4] 2> 4035761 DEBUG (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 13
[junit4] 2> 4035902 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 4035902 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
[junit4] 2> 4035902 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 4035902 DEBUG (AutoscalingActionExecutor-8439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 7962 ms for event id=4df2f6e3c706daT4vacb6hprqr29sjq30ifo2evn
[junit4] 2> 4035903 INFO (TEST-TestSimComputePlanAction.testNodeAdded-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections:
[junit4] 2> ## * .system
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 2
[junit4] 2> ## activeReplicas 2
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 2
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 2
[junit4] 2> ## minActiveSliceDocs 2
[junit4] 2> ## avgActiveSliceDocs 2
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 11,240
[junit4] 2> ## maxActiveSliceBytes 11,240
[junit4] 2> ## minActiveSliceBytes 11,240
[junit4] 2> ## avgActiveSliceBytes 11,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## * testNodeAdded
[junit4] 2> ## shardsTotal 1
[junit4] 2> ## shardsState {active=1}
[junit4] 2> ## shardsWithoutLeader 0
[junit4] 2> ## totalReplicas 4
[junit4] 2> ## activeReplicas 4
[junit4] 2> ## inactiveReplicas 0
[junit4] 2> ## totalActiveDocs 0
[junit4] 2> ## bufferedDocs 0
[junit4] 2> ## maxActiveSliceDocs 0
[junit4] 2> ## minActiveSliceDocs 0
[junit4] 2> ## avgActiveSliceDocs 0
[junit4] 2> ## totalInactiveDocs 0
[junit4] 2> ## maxInactiveSliceDocs 0
[junit4] 2> ## minInactiveSliceDocs 0
[junit4] 2> ## avgInactiveSliceDocs NaN
[junit4] 2> ## totalActiveBytes 10,240
[junit4] 2> ## maxActiveSliceBytes 10,240
[junit4] 2> ## minActiveSliceBytes 10,240
[junit4] 2> ## avgActiveSliceBytes 10,240
[junit4] 2> ## totalInactiveBytes 0
[junit4] 2> ## maxInactiveSliceBytes 0
[junit4] 2> ## minInactiveSliceBytes 0
[junit4] 2> ## avgInactiveSliceBytes NaN
[junit4] 2> ## totalActiveDeletedDocs 0
[junit4] 2> ## Max replicas per node: 4
[junit4] 2> ## Min replicas per node: 2
[junit4] 2> ## Total replicas: 6
[junit4] 2> ## * .system 2
[junit4] 2> ## - active 2
[junit4] 2> ## * testNodeAdded 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 5
[junit4] 2> ## - update 4
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2> ## - AFTER_ACTION 1
[junit4] 2> ## - BEFORE_ACTION 1
[junit4] 2>
[junit4] 2> 4036061 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 4036105 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4036135 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 4036137 DEBUG (ScheduledTrigger-8438-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 4036168 INFO (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
[junit4] 2> 4036168 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 4036168 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=14
[junit4] 2> 4036169 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 14
[junit4] 2> 4036172 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036172 DEBUG (ScheduledTrigger-8438-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10007_solr
[junit4] 2> 4036172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 4036172 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10007_solr
[junit4] 2> 4036172 INFO (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 4036188 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 4036188 DEBUG (ScheduledTrigger-8438-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 4036196 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 4036196 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4036196 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 4036208 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 4036208 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 4036208 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 4036208 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 4036210 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 4036210 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 4036220 WARN (simCloudManagerPool-8437-thread-19) [ ] o.a.s.c.a.s.SimClusterStateProvider Can't find any active replicas for .system / shard1: shard1:{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":4,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":12240,
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.1399388313293457E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":4},
[junit4] 2> "core_node2":{
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":4,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":12240,
[junit4] 2> "node_name":"127.0.0.1:10007_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":1.1399388313293457E-5,
[junit4] 2> "SEARCHER.searcher.numDocs":4}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 4036220 DEBUG (simCloudManagerPool-8437-thread-19) [ ] o.a.s.c.a.s.SimClusterStateProvider -- liveNodes: [127.0.0.1:10006_solr]
[junit4] 2> 4036347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 4036347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 4036347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
[junit4] 2> 4036347 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 4036349 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10006_solr]
[junit4] 2> 4036350 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4036350 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 4036352 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036403 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036440 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036440 DEBUG (simCloudManagerPool-8440-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=15
[junit4] 2> 4036441 DEBUG (simCloudManagerPool-8440-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 15
[junit4] 2> 4036441 DEBUG (simCloudManagerPool-8440-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4036441 DEBUG (simCloudManagerPool-8440-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 4036460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 4036460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4036460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 4036464 DEBUG (simCloudManagerPool-8440-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4036503 DEBUG (simCloudManagerPool-8440-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 4036552 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 4036553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4036553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 4036582 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036603 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036623 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036644 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 4036656 DEBUG (simCloudManagerPool-8440-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 4036657 DEBUG (simCloudManagerPool-8440-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 4036657 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 4036657 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10006_solr, 127.0.0.1:10010_solr, 127.0.0.1:10009_solr, 127.0.0.1:10008_solr]
[junit4] 2> 4036657 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 4036657 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 4036657 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4036671 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4036740 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection testNodeWithMultipleReplicasLost, currentVersion=16
[junit4] 2> 4036741 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=16
[junit4] 2> 4036741 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 16
[junit4] 2> 4036741 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeWithMultipleReplicasLost
[junit4] 2> 4036756 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4036756 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4036760 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=17
[junit4] 2> 4036761 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 17
[junit4] 2> 4036761 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 4036761 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 4036763 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 21940774009 sessionWrapper.createTime 21940773906743220, this.sessionWrapper.createTime 21940773906743220
[junit4] 2> 4036763 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 4036794 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4036806 DEBUG (simCloudManagerPool-8440-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection testNodeWithMultipleReplicasLost, currentVersion=18
[junit4] 2> 4036875 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4036875 DEBUG (simCloudManagerPool-8440-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=18
[junit4] 2> 4036875 DEBUG (simCloudManagerPool-8440-thread-5) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 18
[junit4] 2> 4036894 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037030 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037030 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037030 DEBUG (simCloudManagerPool-8440-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=19
[junit4] 2> 4037030 DEBUG (simCloudManagerPool-8440-thread-6) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 19
[junit4] 2> 4037037 DEBUG (simCloudManagerPool-8440-thread-11) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard1 (currentVersion=20): {"core_node1":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost",
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 4037053 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037053 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037073 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037073 DEBUG (simCloudManagerPool-8440-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=20
[junit4] 2> 4037073 DEBUG (ScheduledTrigger-8441-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037092 DEBUG (simCloudManagerPool-8440-thread-7) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 20
[junit4] 2> 4037094 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037094 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037110 DEBUG (simCloudManagerPool-8440-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=21
[junit4] 2> 4037161 DEBUG (simCloudManagerPool-8440-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 21
[junit4] 2> 4037171 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037171 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037171 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider -- simCreateCollection .system, currentVersion=22
[junit4] 2> 4037172 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=22
[junit4] 2> 4037214 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 22
[junit4] 2> 4037214 DEBUG (simCloudManagerPool-8440-thread-13) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard2 (currentVersion=22): {"core_node4":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost",
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 4037214 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
[junit4] 2> 4037214 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037214 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037224 DEBUG (simCloudManagerPool-8440-thread-9) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=23
[junit4] 2> 4037244 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037244 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037244 DEBUG (simCloudManagerPool-8440-thread-9) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 23
[junit4] 2> 4037247 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=24
[junit4] 2> 4037256 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 24
[junit4] 2> 4037264 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037264 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037267 DEBUG (simCloudManagerPool-8440-thread-10) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=25
[junit4] 2> 4037267 DEBUG (simCloudManagerPool-8440-thread-10) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 25
[junit4] 2> 4037267 DEBUG (MetricsHistoryHandler-8436-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=26
[junit4] 2> 4037270 DEBUG (MetricsHistoryHandler-8436-thread-1) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 26
[junit4] 2> 4037270 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=27
[junit4] 2> 4037270 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 27
[junit4] 2> 4037270 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 4037271 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 4037272 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 21940799450 sessionWrapper.createTime 21940799383799770, this.sessionWrapper.createTime 21940799383799770
[junit4] 2> 4037272 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 4037284 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037284 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037299 DEBUG (simCloudManagerPool-8440-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=28
[junit4] 2> 4037299 DEBUG (simCloudManagerPool-8440-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 28
[junit4] 2> 4037299 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=29
[junit4] 2> 4037316 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037316 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037317 DEBUG (simCloudManagerPool-8440-thread-18) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1 (currentVersion=29): {"core_node3":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}}
[junit4] 2> 4037317 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 29
[junit4] 2> 4037317 DEBUG (TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]) [ ] o.a.s.c.a.s.TestSimComputePlanAction -- cluster state: znodeVersion: 30
[junit4] 2> live nodes:[127.0.0.1:10006_solr, 127.0.0.1:10010_solr, 127.0.0.1:10008_solr, 127.0.0.1:10009_solr]
[junit4] 2> collections:{.system=DocCollection(.system//clusterstate.json/29)={
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}, testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/29)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10009_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10010_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10008_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10009_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10010_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}}
[junit4] 2> 4037327 DEBUG (simCloudManagerPool-8440-thread-16) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=30
[junit4] 2> 4037328 DEBUG (simCloudManagerPool-8440-thread-16) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 30
[junit4] 2> 4037336 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 4037336 DEBUG (ScheduledTrigger-8441-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 4037338 DEBUG (simCloudManagerPool-8440-thread-17) [ ] o.a.s.c.a.s.SimClusterStateProvider ** creating new collection states, currentVersion=31
[junit4] 2> 4037340 DEBUG (simCloudManagerPool-8440-thread-17) [ ] o.a.s.c.a.s.SimClusterStateProvider ** saved cluster state version 31
[junit4] 2> 4037353 DEBUG (simCloudManagerPool-8440-thread-14) [ ] o.a.s.c.a.s.SimClusterStateProvider -- finished createCollection .system, currentVersion=32
[junit4] 2> 4037356 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 4037356 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
[junit4] 2> 4037356 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 4037356 DEBUG (ScheduledTrigger-8441-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
[junit4] 2> 4037376 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 4037376 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 4037376 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10010_solr]
[junit4] 2> 4037376 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4df30b6fdd4206T4vacb6hprqr29sjq30ifo2evs",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":21940803653681670,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[21940803653681670],
[junit4] 2> "preferredOperation":"movereplica",
[junit4] 2> "nodeNames":["127.0.0.1:10010_solr"]}}
[junit4] 2> 4037405 DEBUG (ScheduledTrigger-8441-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
[junit4] 2> ???????? ??, ???? ?:??:?? ??????? com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.solr.cloud.autoscaling.sim.TestSimComputePlanAction
[junit4] 2> ==== jstack at approximately timeout time ====
[junit4] 2> "simCloudManagerPool-8440-thread-19" ID=21331 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@58ee4dd1 owned by "TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]" ID=21274
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.ReentrantLock$NonfairSync@58ee4dd1
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getCollectionStates(SimClusterStateProvider.java:2077)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getClusterState(SimClusterStateProvider.java:2065)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simUpdate(SimClusterStateProvider.java:1401)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:739)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:651)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager$$Lambda$1087/1103631209.call(Unknown Source)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/145500941.run(Unknown Source)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Locked synchronizers:
[junit4] 2> - java.util.concurrent.ThreadPoolExecutor$Worker@17f0e908
[junit4] 2>
[junit4] 2> "simCloudManagerPool-8440-thread-18" ID=21330 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> "simCloudManagerPool-8440-thread-17" ID=21329 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> "simCloudManagerPool-8440-thread-16" ID=21328 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> "simCloudManagerPool-8440-thread-15" ID=21327 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a9362df
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> "simCloudManagerPool-8440-thread-14" ID=21326 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@58ee4dd1 owned by "TEST-TestSimComputePlanAction.testNodeWithMultipleReplicasLost-seed#[B64EC5761226FE96]" ID=21274
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> - waiting on java.util.concurrent.locks.ReentrantLock$NonfairSync@58ee4dd1
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.getCollectionStates(SimClusterStateProvider.java:2077)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimClus
[...truncated too long message...]
ty disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null1132465576
[copy] Copying 239 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null1132465576
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null1132465576
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 93ms :: artifacts dl 17ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 304 minutes 44 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)