You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/05/30 02:21:54 UTC

[JENKINS] Lucene-Solr-Tests-7.x - Build # 628 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/628/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
.response[6][id][0]:100!=17

Stack Trace:
junit.framework.AssertionFailedError: .response[6][id][0]:100!=17
	at __randomizedtesting.SeedInfo.seed([850B1C11A690774B:D5F23CB086C1AB3]:0)
	at junit.framework.Assert.fail(Assert.java:50)
	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:940)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:950)
	at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:102)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13305 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/init-core-data-001
   [junit4]   2> 960677 WARN  (SUITE-BasicDistributedZk2Test-seed#[850B1C11A690774B]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=62 numCloses=62
   [junit4]   2> 960677 INFO  (SUITE-BasicDistributedZk2Test-seed#[850B1C11A690774B]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 960691 INFO  (SUITE-BasicDistributedZk2Test-seed#[850B1C11A690774B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 960692 INFO  (SUITE-BasicDistributedZk2Test-seed#[850B1C11A690774B]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 960692 INFO  (SUITE-BasicDistributedZk2Test-seed#[850B1C11A690774B]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /c/ze
   [junit4]   2> 960711 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 960727 INFO  (Thread-1444) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 960727 INFO  (Thread-1444) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 960765 ERROR (Thread-1444) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 960827 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkTestServer start zk server on port:55917
   [junit4]   2> 960868 INFO  (zkConnectionManagerCallback-1787-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 960899 INFO  (zkConnectionManagerCallback-1789-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 960904 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 960906 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema-sql.xml to /configs/conf1/schema.xml
   [junit4]   2> 960908 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 960909 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 960911 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 960912 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 960913 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 960914 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 960915 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 960917 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 960918 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 960920 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 961780 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 961781 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 961781 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 961781 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 961783 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@431dbd54{/c/ze,null,AVAILABLE}
   [junit4]   2> 961784 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@484bd6bd{HTTP/1.1,[http/1.1]}{127.0.0.1:37564}
   [junit4]   2> 961784 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server Started @961845ms
   [junit4]   2> 961784 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/tempDir-001/control/data, replicaType=NRT, hostContext=/c/ze, hostPort=37564, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/control-001/cores}
   [junit4]   2> 961785 ERROR (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 961785 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 961785 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 961785 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 961785 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 961785 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-30T01:09:26.905Z
   [junit4]   2> 961920 INFO  (zkConnectionManagerCallback-1791-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 961967 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 961967 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/control-001/solr.xml
   [junit4]   2> 961971 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 961971 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 961973 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 962156 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55917/solr
   [junit4]   2> 962179 INFO  (zkConnectionManagerCallback-1795-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 962233 INFO  (zkConnectionManagerCallback-1797-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 962890 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 962891 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37564_c%2Fze
   [junit4]   2> 962892 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.Overseer Overseer (id=73885685558476804-127.0.0.1:37564_c%2Fze-n_0000000000) starting
   [junit4]   2> 962901 INFO  (zkConnectionManagerCallback-1804-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 962904 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 962937 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37564_c%2Fze
   [junit4]   2> 962964 INFO  (zkCallback-1796-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 963049 INFO  (zkCallback-1803-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 963868 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 963915 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 963927 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 963929 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/control-001/cores
   [junit4]   2> 964022 INFO  (zkConnectionManagerCallback-1809-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 964023 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 964024 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 964040 INFO  (qtp445011956-6652) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37564_c%252Fze&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 964070 INFO  (OverseerThreadFactory-2933-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 964200 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze    x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 964201 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze    x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 965330 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 965543 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 965863 WARN  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 965863 WARN  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 965863 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 965863 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 965864 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 965864 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 965864 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 965887 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1640100898, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 965918 WARN  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 966079 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 966079 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 966089 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 966090 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 966092 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 966095 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62ca5add[control_collection_shard1_replica_n1] main]
   [junit4]   2> 966097 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 966097 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 966098 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 966100 INFO  (searcherExecutor-2936-thread-1-processing-n:127.0.0.1:37564_c%2Fze x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@62ca5add[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 966102 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601849336761679872
   [junit4]   2> 966124 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37564/c/ze/control_collection_shard1_replica_n1/
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37564/c/ze/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 966131 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 966158 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37564/c/ze/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 966261 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 966264 INFO  (qtp445011956-6647) [n:127.0.0.1:37564_c%2Fze c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2064
   [junit4]   2> 966268 INFO  (qtp445011956-6652) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 966364 INFO  (zkCallback-1796-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 967269 INFO  (qtp445011956-6652) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37564_c%252Fze&wt=javabin&version=2} status=0 QTime=3229
   [junit4]   2> 967323 INFO  (zkConnectionManagerCallback-1814-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 967324 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 967325 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 967325 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 967326 INFO  (qtp445011956-6648) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 967346 INFO  (OverseerCollectionConfigSetProcessor-73885685558476804-127.0.0.1:37564_c%2Fze-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 967347 INFO  (OverseerThreadFactory-2933-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 967348 WARN  (OverseerThreadFactory-2933-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 967635 INFO  (qtp445011956-6648) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 967635 INFO  (qtp445011956-6648) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=308
   [junit4]   2> 967906 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001 of type NRT
   [junit4]   2> 967907 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 967949 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 967949 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 967949 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 967949 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@119584fb{/c/ze,null,AVAILABLE}
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@13235aa4{HTTP/1.1,[http/1.1]}{127.0.0.1:47353}
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server Started @968010ms
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/c/ze, hostPort=47353, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001/cores}
   [junit4]   2> 967950 ERROR (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 967950 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 967951 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 967951 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-30T01:09:33.071Z
   [junit4]   2> 967994 INFO  (zkConnectionManagerCallback-1816-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 967995 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 967995 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001/solr.xml
   [junit4]   2> 967999 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 967999 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 968001 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 968014 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55917/solr
   [junit4]   2> 968016 INFO  (zkConnectionManagerCallback-1820-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 968018 INFO  (zkConnectionManagerCallback-1822-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 968026 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 968029 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 968035 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 968035 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47353_c%2Fze
   [junit4]   2> 968037 INFO  (zkCallback-1796-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 968039 INFO  (zkCallback-1803-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 968039 INFO  (zkCallback-1813-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 968083 INFO  (zkCallback-1821-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 968218 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 968234 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 968234 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 968237 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001/cores
   [junit4]   2> 968241 INFO  (zkConnectionManagerCallback-1829-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 968242 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 968243 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:47353_c%2Fze    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 968279 INFO  (qtp445011956-6654) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:47353_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 968282 INFO  (OverseerCollectionConfigSetProcessor-73885685558476804-127.0.0.1:37564_c%2Fze-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 968287 INFO  (OverseerThreadFactory-2933-thread-3) [ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:47353_c%2Fze for creating new replica
   [junit4]   2> 968295 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze    x:collection1_shard2_replica_n41] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 969332 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 969401 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 969834 WARN  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 969835 WARN  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 969847 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 969847 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 969848 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 969848 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 969848 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 969876 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1640100898, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 969890 WARN  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 970256 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 970256 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 970259 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 970259 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 970261 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 970262 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@77dce1d2[collection1_shard2_replica_n41] main]
   [junit4]   2> 970263 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 970264 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 970265 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 970265 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601849341126901760
   [junit4]   2> 970268 INFO  (searcherExecutor-2947-thread-1-processing-n:127.0.0.1:47353_c%2Fze x:collection1_shard2_replica_n41 c:collection1 s:shard2) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@77dce1d2[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 970286 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node42=0}, version=0}
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47353/c/ze/collection1_shard2_replica_n41/
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:47353/c/ze/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 970289 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 970293 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47353/c/ze/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 970443 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 970447 INFO  (qtp1171230627-6706) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2151
   [junit4]   2> 970450 INFO  (qtp445011956-6654) [n:127.0.0.1:37564_c%2Fze c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:47353_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=2170
   [junit4]   2> 970606 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001 of type NRT
   [junit4]   2> 970607 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 970608 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 970608 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 970608 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 970608 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ec83330{/c/ze,null,AVAILABLE}
   [junit4]   2> 970609 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@43960364{HTTP/1.1,[http/1.1]}{127.0.0.1:45403}
   [junit4]   2> 970609 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server Started @970669ms
   [junit4]   2> 970609 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/c/ze, hostPort=45403, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001/cores}
   [junit4]   2> 970609 ERROR (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 970616 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 970616 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 970616 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 970616 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 970616 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-30T01:09:35.736Z
   [junit4]   2> 970619 INFO  (zkConnectionManagerCallback-1831-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 970620 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 970620 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001/solr.xml
   [junit4]   2> 970624 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 970624 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 970626 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 970654 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55917/solr
   [junit4]   2> 970656 INFO  (zkConnectionManagerCallback-1835-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 970658 INFO  (zkConnectionManagerCallback-1837-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 970665 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 970674 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 970677 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 970677 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45403_c%2Fze
   [junit4]   2> 970679 INFO  (zkCallback-1803-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970679 INFO  (zkCallback-1813-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970679 INFO  (zkCallback-1828-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970679 INFO  (zkCallback-1821-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970679 INFO  (zkCallback-1796-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970687 INFO  (zkCallback-1836-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 970924 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 970957 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 970958 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 970972 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001/cores
   [junit4]   2> 971003 INFO  (zkConnectionManagerCallback-1844-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 971004 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 971006 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45403_c%2Fze    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 971087 INFO  (qtp445011956-6650) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45403_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 971136 INFO  (OverseerCollectionConfigSetProcessor-73885685558476804-127.0.0.1:37564_c%2Fze-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 971148 INFO  (OverseerThreadFactory-2933-thread-4) [ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45403_c%2Fze for creating new replica
   [junit4]   2> 971152 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze    x:collection1_shard1_replica_n43] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 972169 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 972185 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 972314 WARN  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 972314 WARN  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 972314 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 972314 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 972315 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 972315 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 972315 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 972318 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1640100898, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 972322 WARN  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 972384 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 972384 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 972385 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 972385 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 972387 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 972388 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@63ad239f[collection1_shard1_replica_n43] main]
   [junit4]   2> 972389 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 972391 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 972392 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 972393 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601849343358271488
   [junit4]   2> 972397 INFO  (searcherExecutor-2958-thread-1-processing-n:127.0.0.1:45403_c%2Fze x:collection1_shard1_replica_n43 c:collection1 s:shard1) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@63ad239f[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 972400 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node44=0}, version=0}
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45403/c/ze/collection1_shard1_replica_n43/
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:45403/c/ze/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 972403 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 972407 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45403/c/ze/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 972558 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 972562 INFO  (qtp852512254-6742) [n:127.0.0.1:45403_c%2Fze c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1410
   [junit4]   2> 972565 INFO  (qtp445011956-6650) [n:127.0.0.1:37564_c%2Fze c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45403_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1477
   [junit4]   2> 972866 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001 of type NRT
   [junit4]   2> 972867 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 972868 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 972868 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 972868 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 972868 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10f7fa03{/c/ze,null,AVAILABLE}
   [junit4]   2> 972869 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@58dc4576{HTTP/1.1,[http/1.1]}{127.0.0.1:45568}
   [junit4]   2> 972869 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.e.j.s.Server Started @972929ms
   [junit4]   2> 972869 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/c/ze, hostPort=45568, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001/cores}
   [junit4]   2> 972869 ERROR (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 972919 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 972919 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 972919 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 972919 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 972920 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-30T01:09:38.039Z
   [junit4]   2> 972963 INFO  (zkConnectionManagerCallback-1846-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 972973 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 972973 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001/solr.xml
   [junit4]   2> 972977 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 972977 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 972979 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 973008 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55917/solr
   [junit4]   2> 973027 INFO  (zkConnectionManagerCallback-1850-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 973070 INFO  (zkConnectionManagerCallback-1852-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 973077 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 973080 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 973083 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 973083 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45568_c%2Fze
   [junit4]   2> 973085 INFO  (zkCallback-1813-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973085 INFO  (zkCallback-1828-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973086 INFO  (zkCallback-1803-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973086 INFO  (zkCallback-1821-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973086 INFO  (zkCallback-1836-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973086 INFO  (zkCallback-1843-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973086 INFO  (zkCallback-1796-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973115 INFO  (zkCallback-1851-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 973138 INFO  (OverseerCollectionConfigSetProcessor-73885685558476804-127.0.0.1:37564_c%2Fze-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 973388 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 973406 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 973407 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 973409 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001/cores
   [junit4]   2> 973413 INFO  (zkConnectionManagerCallback-1859-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 973414 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 973416 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [n:127.0.0.1:45568_c%2Fze    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55917/solr ready
   [junit4]   2> 973481 INFO  (qtp445011956-6654) [n:127.0.0.1:37564_c%2Fze    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45568_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 973485 INFO  (OverseerThreadFactory-2933-thread-5) [ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45568_c%2Fze for creating new replica
   [junit4]   2> 973490 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze    x:collection1_shard2_replica_n45] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 974511 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 974527 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
   [junit4]   2> 974622 WARN  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 974622 WARN  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 974622 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 974623 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
   [junit4]   2> 974623 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'solr.core.collection1.shard2.replica_n45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@652ba812
   [junit4]   2> 974623 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 974624 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [[collection1_shard2_replica_n45] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001/cores/collection1_shard2_replica_n45], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
   [junit4]   2> 974627 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1640100898, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 974632 WARN  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 974695 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 974695 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 974696 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 974697 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 974698 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=22, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 974699 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.SolrIndexSearcher Opening [Searcher@5cce3c5d[collection1_shard2_replica_n45] main]
   [junit4]   2> 974700 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 974700 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 974701 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 974701 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1601849345778384896
   [junit4]   2> 974706 INFO  (searcherExecutor-2969-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher Searcher@5cce3c5d[collection1_shard2_replica_n45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 974713 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node42=0, core_node46=0}, version=1}
   [junit4]   2> 974714 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n45
   [junit4]   2> 974716 INFO  (qtp621087600-6778) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1226
   [junit4]   2> 974719 INFO  (qtp445011956-6654) [n:127.0.0.1:37564_c%2Fze c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45568_c%252Fze&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1238
   [junit4]   2> 974721 INFO  (updateExecutor-1847-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 974758 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 974759 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 974775 INFO  (qtp1171230627-6701) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2 r:core_node42 x:collection1_shard2_replica_n41] o.a.s.c.S.Request [collection1_shard2_replica_n41]  webapp=/c/ze path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 974775 INFO  (qtp1171230627-6701) [n:127.0.0.1:47353_c%2Fze c:collection1 s:shard2 r:core_node42 x:collection1_shard2_replica_n41] o.a.s.c.S.Request [collection1_shard2_replica_n41]  webapp=/c/ze path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 974775 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n45]
   [junit4]   2> 974775 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 974776 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n45] as recovering, leader is [http://127.0.0.1:47353/c/ze/collection1_shard2_replica_n41/] and I am [http://127.0.0.1:45568/c/ze/collection1_shard2_replica_n45/]
   [junit4]   2> 974794 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node42=0, core_node46=0, core_node46_recovering=0}, version=2}
   [junit4]   2> 974797 INFO  (recoveryExecutor-1848-thread-1-processing-n:127.0.0.1:45568_c%2Fze x:collection1_shard2_replica_n45 c:collection1 s:shard2 r:core_node46) [n:127.0.0.1:45568_c%2Fze c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:47353/c/ze]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:45568_c%252Fze&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 974798 INFO  (qtp1171230627-6702) [n:127.0.0.1:47353_c%2Fze    x:collection1_shard2_replica_n41] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 974811 INFO  (qtp1171230627-6702) [n:127.0.0.1:47353_c%2Fze    x:collection1_shard2_replica_n41] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:45568_c%2Fze, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:45568/c/ze","node_name":"127.0.0.1:45568_c%2Fze","state":"down","type":"NRT"}
   [junit4]   2> 974924 INFO  (TEST-BasicDistributedZk2Test.test-seed#[850B1C11A690774B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.BasicDistributedZk2Test_850B1C11A690774B-001/shard-4-001 of type NRT
   [junit4]   2> 974925 INFO  (TEST

[...truncated too long message...]

-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1763571701
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1763571701
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1763571701

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null327849625
     [copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null327849625
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null327849625

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 73ms :: artifacts dl 8ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 148 minutes 1 second
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-Tests-7.x - Build # 631 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/631/

2 tests failed.
FAILED:  org.apache.solr.cloud.CleanupOldIndexTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([DEB32B10C88955D2]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CleanupOldIndexTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([DEB32B10C88955D2]:0)




Build Log:
[...truncated 14995 lines...]
   [junit4] Suite: org.apache.solr.cloud.CleanupOldIndexTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/init-core-data-001
   [junit4]   2> 351929 WARN  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
   [junit4]   2> 351930 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 351931 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 351931 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 351932 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001
   [junit4]   2> 351932 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 351943 INFO  (Thread-411) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 351943 INFO  (Thread-411) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 351964 ERROR (Thread-411) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 352044 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:56944
   [junit4]   2> 352079 INFO  (zkConnectionManagerCallback-691-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352122 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 352140 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 352333 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 352333 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 352333 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 352343 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@247e4eba{/solr,null,AVAILABLE}
   [junit4]   2> 352345 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1bae2a3e{HTTP/1.1,[http/1.1]}{127.0.0.1:46574}
   [junit4]   2> 352345 INFO  (jetty-launcher-688-thread-2) [    ] o.e.j.s.Server Started @352530ms
   [junit4]   2> 352345 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46574}
   [junit4]   2> 352346 ERROR (jetty-launcher-688-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 352346 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 352346 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 352346 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 352346 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 352346 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-02T10:26:32.834Z
   [junit4]   2> 352356 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 352356 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 352356 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 352356 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e59d88d{/solr,null,AVAILABLE}
   [junit4]   2> 352357 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@781bdfc8{HTTP/1.1,[http/1.1]}{127.0.0.1:50273}
   [junit4]   2> 352357 INFO  (jetty-launcher-688-thread-1) [    ] o.e.j.s.Server Started @352542ms
   [junit4]   2> 352357 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50273}
   [junit4]   2> 352357 ERROR (jetty-launcher-688-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 352357 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 352358 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 352358 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 352358 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 352358 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-02T10:26:32.846Z
   [junit4]   2> 352387 INFO  (zkConnectionManagerCallback-695-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352388 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 352399 INFO  (zkConnectionManagerCallback-693-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352403 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 352503 INFO  (jetty-launcher-688-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56944/solr
   [junit4]   2> 352549 INFO  (jetty-launcher-688-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56944/solr
   [junit4]   2> 352550 INFO  (zkConnectionManagerCallback-699-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352561 INFO  (zkConnectionManagerCallback-703-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352639 INFO  (zkConnectionManagerCallback-705-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352700 INFO  (zkConnectionManagerCallback-707-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352832 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 352833 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50273_solr
   [junit4]   2> 352840 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.Overseer Overseer (id=73904863133827078-127.0.0.1:50273_solr-n_0000000000) starting
   [junit4]   2> 352859 INFO  (zkConnectionManagerCallback-716-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 352874 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56944/solr ready
   [junit4]   2> 352887 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50273_solr
   [junit4]   2> 352934 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 352942 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 352975 INFO  (zkCallback-715-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 353046 INFO  (zkCallback-706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 353056 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 353056 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46574_solr
   [junit4]   2> 353061 INFO  (zkCallback-706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 353065 INFO  (zkCallback-715-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 353089 INFO  (zkCallback-704-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 353167 WARN  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 353218 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50273.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353240 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50273.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353241 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50273.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353242 INFO  (jetty-launcher-688-thread-1) [n:127.0.0.1:50273_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node1/.
   [junit4]   2> 353559 INFO  (zkConnectionManagerCallback-723-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 353561 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 353562 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56944/solr ready
   [junit4]   2> 353562 WARN  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 353590 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46574.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353601 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46574.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353601 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46574.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 353603 INFO  (jetty-launcher-688-thread-2) [n:127.0.0.1:46574_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node2/.
   [junit4]   2> 353683 INFO  (zkConnectionManagerCallback-725-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 353688 INFO  (zkConnectionManagerCallback-730-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 353690 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 353691 INFO  (SUITE-CleanupOldIndexTest-seed#[DEB32B10C88955D2]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56944/solr ready
   [junit4]   2> 353729 INFO  (TEST-CleanupOldIndexTest.test-seed#[DEB32B10C88955D2]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 353732 INFO  (qtp939978042-2382) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=eef9cfe5-3ce8-45e8-be67-67c359ea2042&replicationFactor=2&collection.configName=conf1&name=oldindextest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 353740 INFO  (qtp939978042-2382) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=eef9cfe5-3ce8-45e8-be67-67c359ea2042&replicationFactor=2&collection.configName=conf1&name=oldindextest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 353742 INFO  (qtp939978042-2378) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 353744 INFO  (qtp939978042-2378) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 353753 INFO  (OverseerThreadFactory-826-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection oldindextest
   [junit4]   2> 353865 INFO  (OverseerStateUpdate-73904863133827078-127.0.0.1:50273_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"oldindextest",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"oldindextest_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:50273/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 353874 INFO  (OverseerStateUpdate-73904863133827078-127.0.0.1:50273_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"oldindextest",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"oldindextest_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:46574/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 354093 INFO  (qtp933356857-2375) [n:127.0.0.1:50273_solr    x:oldindextest_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=oldindextest&version=2&replicaType=NRT&async=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&coreNodeName=core_node2&name=oldindextest_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=5
   [junit4]   2> 354095 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr    x:oldindextest_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=oldindextest&version=2&replicaType=NRT&async=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&coreNodeName=core_node2&name=oldindextest_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 354095 INFO  (qtp939978042-2377) [n:127.0.0.1:46574_solr    x:oldindextest_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=oldindextest&version=2&replicaType=NRT&async=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624&coreNodeName=core_node4&name=oldindextest_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=8
   [junit4]   2> 354096 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr    x:oldindextest_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 354099 INFO  (qtp933356857-2371) [n:127.0.0.1:50273_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574
   [junit4]   2> 354096 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr    x:oldindextest_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=oldindextest&version=2&replicaType=NRT&async=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624&coreNodeName=core_node4&name=oldindextest_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 354102 INFO  (qtp933356857-2371) [n:127.0.0.1:50273_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 354745 INFO  (qtp939978042-2378) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 354748 INFO  (qtp939978042-2378) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 355103 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574
   [junit4]   2> 355103 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 355121 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 355132 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
   [junit4]   2> 355150 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.s.IndexSchema [oldindextest_shard1_replica_n3] Schema name=cloud-dynamic
   [junit4]   2> 355161 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.s.IndexSchema [oldindextest_shard1_replica_n1] Schema name=cloud-dynamic
   [junit4]   2> 355290 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 355290 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'oldindextest_shard1_replica_n3' using configuration from collection oldindextest, trusted=true
   [junit4]   2> 355303 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46574.solr.core.oldindextest.shard1.replica_n3' (registry 'solr.core.oldindextest.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 355307 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 355308 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'oldindextest_shard1_replica_n1' using configuration from collection oldindextest, trusted=true
   [junit4]   2> 355308 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50273.solr.core.oldindextest.shard1.replica_n1' (registry 'solr.core.oldindextest.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445dc874
   [junit4]   2> 355321 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 355336 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 355446 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.SolrCore [[oldindextest_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node2/oldindextest_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node2/./oldindextest_shard1_replica_n3/data/]
   [junit4]   2> 355451 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SolrCore [[oldindextest_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node1/oldindextest_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.CleanupOldIndexTest_DEB32B10C88955D2-001/tempDir-001/node1/./oldindextest_shard1_replica_n1/data/]
   [junit4]   2> 355749 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 355751 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 355811 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 355811 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 355820 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 355820 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 355820 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 355820 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 355822 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 355822 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 355829 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@69e56c05[oldindextest_shard1_replica_n1] main]
   [junit4]   2> 355830 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@3194e3a4[oldindextest_shard1_replica_n3] main]
   [junit4]   2> 355840 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 355841 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 355842 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 355842 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1602156176426926080
   [junit4]   2> 355851 INFO  (searcherExecutor-835-thread-1-processing-x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 r:core_node4 CREATE n:127.0.0.1:46574_solr c:oldindextest s:shard1) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.SolrCore [oldindextest_shard1_replica_n3] Registered new searcher Searcher@3194e3a4[oldindextest_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 355952 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 355958 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 355959 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 355962 INFO  (searcherExecutor-836-thread-1-processing-x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 r:core_node2 CREATE n:127.0.0.1:50273_solr c:oldindextest s:shard1) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SolrCore [oldindextest_shard1_replica_n1] Registered new searcher Searcher@69e56c05[oldindextest_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 355963 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1602156176553803776
   [junit4]   2> 355977 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/oldindextest/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 356002 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9994ms
   [junit4]   2> 356091 INFO  (parallelCoreAdminExecutor-831-thread-1-processing-n:127.0.0.1:46574_solr x:oldindextest_shard1_replica_n3 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624 CREATE) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/oldindextest/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 356105 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574
   [junit4]   2> 356105 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 356503 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 356504 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 356504 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/
   [junit4]   2> 356504 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=oldindextest_shard1_replica_n1 url=http://127.0.0.1:50273/solr START replicas=[http://127.0.0.1:46574/solr/oldindextest_shard1_replica_n3/] nUpdates=100
   [junit4]   2> 356511 INFO  (qtp939978042-2380) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.c.S.Request [oldindextest_shard1_replica_n3]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 356753 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 356754 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 356805 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=oldindextest_shard1_replica_n1 url=http://127.0.0.1:50273/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 356805 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 356805 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 356805 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 356816 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/ shard1
   [junit4]   2> 356956 INFO  (parallelCoreAdminExecutor-827-thread-1-processing-n:127.0.0.1:50273_solr x:oldindextest_shard1_replica_n1 eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574 CREATE) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 357060 INFO  (zkCallback-706-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/oldindextest/state.json] for collection [oldindextest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 357140 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574
   [junit4]   2> 357140 INFO  (qtp933356857-2374) [n:127.0.0.1:50273_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091393578574&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 357145 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624
   [junit4]   2> 357146 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 357250 INFO  (zkCallback-704-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/oldindextest/state.json] for collection [oldindextest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 357250 INFO  (zkCallback-706-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/oldindextest/state.json] for collection [oldindextest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 357756 INFO  (qtp939978042-2380) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 357757 INFO  (qtp939978042-2380) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 358147 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624
   [junit4]   2> 358147 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=eef9cfe5-3ce8-45e8-be67-67c359ea204228187091402788624&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 358151 INFO  (OverseerThreadFactory-826-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 358759 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 358764 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 358765 INFO  (qtp939978042-2380) [n:127.0.0.1:46574_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 358766 INFO  (qtp939978042-2380) [n:127.0.0.1:46574_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=eef9cfe5-3ce8-45e8-be67-67c359ea2042&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 358876 INFO  (qtp933356857-2370) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/oldindextest/terms/shard1 to Terms{values={core_node2=1, core_node4=1}, version=2}
   [junit4]   2> 358895 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-0 (1602156179531759616)]} 0 17
   [junit4]   2> 358896 INFO  (qtp933356857-2370) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-0 (1602156179531759616)]} 0 103
   [junit4]   2> 358925 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-1 (1602156179631374336)]} 0 13
   [junit4]   2> 358925 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-1 (1602156179631374336)]} 0 28
   [junit4]   2> 358943 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-0 (-1602156179661783040)]} 0 11
   [junit4]   2> 358944 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-0 (-1602156179661783040)]} 0 17
   [junit4]   2> 358947 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-2 (1602156179680657408)]} 0 0
   [junit4]   2> 358948 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-2 (1602156179680657408)]} 0 2
   [junit4]   2> 358956 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-1 (-1602156179684851712)]} 0 7
   [junit4]   2> 358957 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-1 (-1602156179684851712)]} 0 8
   [junit4]   2> 358959 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-3 (1602156179694288896)]} 0 0
   [junit4]   2> 358960 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-3 (1602156179694288896)]} 0 2
   [junit4]   2> 358961 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-2 (-1602156179696386048)]} 0 0
   [junit4]   2> 358962 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-2 (-1602156179696386048)]} 0 1
   [junit4]   2> 358964 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-4 (1602156179699531776)]} 0 0
   [junit4]   2> 358965 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-4 (1602156179699531776)]} 0 2
   [junit4]   2> 358968 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-5 (1602156179702677504)]} 0 0
   [junit4]   2> 358968 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-5 (1602156179702677504)]} 0 2
   [junit4]   2> 358970 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-6 (1602156179705823232)]} 0 0
   [junit4]   2> 358971 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-6 (1602156179705823232)]} 0 2
   [junit4]   2> 358974 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-7 (1602156179708968960)]} 0 0
   [junit4]   2> 358974 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-7 (1602156179708968960)]} 0 2
   [junit4]   2> 358977 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-8 (1602156179712114688)]} 0 0
   [junit4]   2> 358977 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-8 (1602156179712114688)]} 0 2
   [junit4]   2> 358980 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-9 (1602156179715260416)]} 0 0
   [junit4]   2> 358980 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-9 (1602156179715260416)]} 0 2
   [junit4]   2> 358982 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-9 (-1602156179718406144)]} 0 0
   [junit4]   2> 358982 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-9 (-1602156179718406144)]} 0 1
   [junit4]   2> 358985 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-10 (1602156179720503296)]} 0 0
   [junit4]   2> 358985 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-10 (1602156179720503296)]} 0 2
   [junit4]   2> 358995 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-11 (1602156179723649024)]} 0 0
   [junit4]   2> 358996 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-11 (1602156179723649024)]} 0 10
   [junit4]   2> 358999 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-12 (1602156179735183360)]} 0 0
   [junit4]   2> 358999 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-12 (1602156179735183360)]} 0 2
   [junit4]   2> 359001 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-10 (-1602156179738329088)]} 0 0
   [junit4]   2> 359001 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-10 (-1602156179738329088)]} 0 1
   [junit4]   2> 359003 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-13 (1602156179740426240)]} 0 0
   [junit4]   2> 359004 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-13 (1602156179740426240)]} 0 2
   [junit4]   2> 359006 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-12 (-1602156179743571968)]} 0 0
   [junit4]   2> 359006 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-12 (-1602156179743571968)]} 0 1
   [junit4]   2> 359014 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-14 (1602156179750912000)]} 0 0
   [junit4]   2> 359015 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-14 (1602156179750912000)]} 0 2
   [junit4]   2> 359025 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-15 (1602156179762446336)]} 0 0
   [junit4]   2> 359025 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-15 (1602156179762446336)]} 0 2
   [junit4]   2> 359028 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-16 (1602156179765592064)]} 0 0
   [junit4]   2> 359028 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-16 (1602156179765592064)]} 0 2
   [junit4]   2> 359030 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-17 (1602156179768737792)]} 0 0
   [junit4]   2> 359031 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-17 (1602156179768737792)]} 0 2
   [junit4]   2> 359033 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-18 (1602156179771883520)]} 0 0
   [junit4]   2> 359033 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-18 (1602156179771883520)]} 0 2
   [junit4]   2> 359035 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-13 (-1602156179773980672)]} 0 0
   [junit4]   2> 359036 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-13 (-1602156179773980672)]} 0 1
   [junit4]   2> 359038 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-19 (1602156179776077824)]} 0 0
   [junit4]   2> 359039 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-19 (1602156179776077824)]} 0 2
   [junit4]   2> 359046 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-20 (1602156179781320704)]} 0 2
   [junit4]   2> 359047 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-20 (1602156179781320704)]} 0 6
   [junit4]   2> 359053 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-21 (1602156179791806464)]} 0 1
   [junit4]   2> 359054 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-21 (1602156179791806464)]} 0 6
   [junit4]   2> 359056 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-14 (-1602156179794952192)]} 0 0
   [junit4]   2> 359056 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-14 (-1602156179794952192)]} 0 1
   [junit4]   2> 359067 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-22 (1602156179798097920)]} 0 1
   [junit4]   2> 359067 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-22 (1602156179798097920)]} 0 10
   [junit4]   2> 359074 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-23 (1602156179813826560)]} 0 0
   [junit4]   2> 359075 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-23 (1602156179813826560)]} 0 2
   [junit4]   2> 359077 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-24 (1602156179818020864)]} 0 0
   [junit4]   2> 359078 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-24 (1602156179818020864)]} 0 2
   [junit4]   2> 359083 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-25 (1602156179821166592)]} 0 1
   [junit4]   2> 359084 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-25 (1602156179821166592)]} 0 5
   [junit4]   2> 359087 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-26 (1602156179827458048)]} 0 0
   [junit4]   2> 359087 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-26 (1602156179827458048)]} 0 2
   [junit4]   2> 359090 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-27 (1602156179830603776)]} 0 1
   [junit4]   2> 359091 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-27 (1602156179830603776)]} 0 2
   [junit4]   2> 359095 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-28 (1602156179835846656)]} 0 0
   [junit4]   2> 359095 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-28 (1602156179835846656)]} 0 2
   [junit4]   2> 359103 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-29 (1602156179838992384)]} 0 4
   [junit4]   2> 359104 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-29 (1602156179838992384)]} 0 7
   [junit4]   2> 359106 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-16 (-1602156179848429568)]} 0 0
   [junit4]   2> 359106 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-16 (-1602156179848429568)]} 0 1
   [junit4]   2> 359144 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-30 (1602156179851575296)]} 0 1
   [junit4]   2> 359145 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-30 (1602156179851575296)]} 0 36
   [junit4]   2> 359152 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-31 (1602156179891421184)]} 0 1
   [junit4]   2> 359153 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-31 (1602156179891421184)]} 0 6
   [junit4]   2> 359160 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-32 (1602156179899809792)]} 0 1
   [junit4]   2> 359161 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-32 (1602156179899809792)]} 0 7
   [junit4]   2> 359170 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-33 (1602156179908198400)]} 0 1
   [junit4]   2> 359171 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-33 (1602156179908198400)]} 0 9
   [junit4]   2> 359173 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-18 (-1602156179918684160)]} 0 0
   [junit4]   2> 359173 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-18 (-1602156179918684160)]} 0 1
   [junit4]   2> 359181 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-34 (1602156179920781312)]} 0 1
   [junit4]   2> 359181 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-34 (1602156179920781312)]} 0 7
   [junit4]   2> 359183 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-20 (-1602156179929169920)]} 0 0
   [junit4]   2> 359184 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-20 (-1602156179929169920)]} 0 2
   [junit4]   2> 359213 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-35 (1602156179932315648)]} 0 1
   [junit4]   2> 359214 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-35 (1602156179932315648)]} 0 28
   [junit4]   2> 359216 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-21 (-1602156179963772928)]} 0 0
   [junit4]   2> 359216 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-21 (-1602156179963772928)]} 0 1
   [junit4]   2> 359218 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-36 (1602156179965870080)]} 0 0
   [junit4]   2> 359219 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-36 (1602156179965870080)]} 0 1
   [junit4]   2> 359221 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-37 (1602156179969015808)]} 0 0
   [junit4]   2> 359221 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-37 (1602156179969015808)]} 0 1
   [junit4]   2> 359223 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-38 (1602156179971112960)]} 0 0
   [junit4]   2> 359224 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-38 (1602156179971112960)]} 0 1
   [junit4]   2> 359225 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-22 (-1602156179974258688)]} 0 0
   [junit4]   2> 359226 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-22 (-1602156179974258688)]} 0 1
   [junit4]   2> 359228 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-39 (1602156179975307264)]} 0 0
   [junit4]   2> 359228 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-39 (1602156179975307264)]} 0 1
   [junit4]   2> 359230 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-23 (-1602156179978452992)]} 0 0
   [junit4]   2> 359230 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-23 (-1602156179978452992)]} 0 1
   [junit4]   2> 359233 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-40 (1602156179980550144)]} 0 0
   [junit4]   2> 359233 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-40 (1602156179980550144)]} 0 1
   [junit4]   2> 359235 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-41 (1602156179983695872)]} 0 0
   [junit4]   2> 359236 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-41 (1602156179983695872)]} 0 1
   [junit4]   2> 359238 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-42 (1602156179985793024)]} 0 0
   [junit4]   2> 359239 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-42 (1602156179985793024)]} 0 2
   [junit4]   2> 359240 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-24 (-1602156179988938752)]} 0 0
   [junit4]   2> 359241 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-24 (-1602156179988938752)]} 0 1
   [junit4]   2> 359243 INFO  (qtp939978042-2383) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-43 (1602156179992084480)]} 0 0
   [junit4]   2> 359243 INFO  (qtp933356857-2376) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-43 (1602156179992084480)]} 0 1
   [junit4]   2> 359246 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{delete=[1-27 (-1602156179994181632)]} 0 0
   [junit4]   2> 359246 INFO  (qtp933356857-2386) [n:127.0.0.1:50273_solr c:oldindextest s:shard1 r:core_node2 x:oldindextest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-27 (-1602156179994181632)]} 0 1
   [junit4]   2> 359248 INFO  (qtp939978042-2381) [n:127.0.0.1:46574_solr c:oldindextest s:shard1 r:core_node4 x:oldindextest_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [oldindextest_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50273/solr/oldindextest_shard1_replica_n1/&wt=javabin&version=2}{add=[1-44 (1602156179997327360)]} 0 0
   [junit4]   2> 359248 INFO  (qtp933356857-2376) [n:127.0.0.1:

[...truncated too long message...]

onfigure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1971115857
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1971115857
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1971115857

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null314346851
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null314346851
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null314346851

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 36ms :: artifacts dl 2ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 240 minutes 50 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-Tests-7.x - Build # 630 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/630/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded

Error Message:
ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@1326cfc9, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@10c50d50] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@1326cfc9, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@10c50d50] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([4B37CA88DAB0C8C7:2EF49CFF781360C4]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/41)={   "replicationFactor":"2",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testSplitIntegration_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10004_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11},         "core_node4":{           "core":"testSplitIntegration_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10005_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1527896376941472600",       "replicas":{         "core_node1":{           "core":"testSplitIntegration_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10004_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14},         "core_node2":{           "core":"testSplitIntegration_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10005_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1527896376976772250",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10005_solr",           "base_url":"http://127.0.0.1:10005/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node9":{           "core":"testSplitIntegration_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10004_solr",           "base_url":"http://127.0.0.1:10004/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1527896376976435050",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10004_solr",           "base_url":"http://127.0.0.1:10004/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node8":{           "core":"testSplitIntegration_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10005_solr",           "base_url":"http://127.0.0.1:10005/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/41)={
  "replicationFactor":"2",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testSplitIntegration_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10004_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11},
        "core_node4":{
          "core":"testSplitIntegration_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10005_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1527896376941472600",
      "replicas":{
        "core_node1":{
          "core":"testSplitIntegration_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10004_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14},
        "core_node2":{
          "core":"testSplitIntegration_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10005_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1527896376976772250",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10005_solr",
          "base_url":"http://127.0.0.1:10005/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node9":{
          "core":"testSplitIntegration_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10004_solr",
          "base_url":"http://127.0.0.1:10004/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1527896376976435050",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10004_solr",
          "base_url":"http://127.0.0.1:10004/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node8":{
          "core":"testSplitIntegration_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10005_solr",
          "base_url":"http://127.0.0.1:10005/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}}}}
	at __randomizedtesting.SeedInfo.seed([4B37CA88DAB0C8C7:72B973C8F54F0139]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
	at __randomizedtesting.SeedInfo.seed([4B37CA88DAB0C8C7:188E883838A15D3D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:404)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14611 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_4B37CA88DAB0C8C7-001/init-core-data-001
   [junit4]   2> 2956957 WARN  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=35 numCloses=35
   [junit4]   2> 2956957 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2956959 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2956972 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2956973 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_4B37CA88DAB0C8C7-001/tempDir-001
   [junit4]   2> 2956973 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2956975 INFO  (Thread-2888) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2956975 INFO  (Thread-2888) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2956976 ERROR (Thread-2888) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2957075 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:50144
   [junit4]   2> 2957083 INFO  (zkConnectionManagerCallback-3519-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957095 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 2957106 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 2957107 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2957107 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2957107 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2957107 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b49ab88{/solr,null,AVAILABLE}
   [junit4]   2> 2957108 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7ebb3bde{SSL,[ssl, http/1.1]}{127.0.0.1:42455}
   [junit4]   2> 2957108 INFO  (jetty-launcher-3516-thread-1) [    ] o.e.j.s.Server Started @2957257ms
   [junit4]   2> 2957108 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42455}
   [junit4]   2> 2957109 ERROR (jetty-launcher-3516-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2957109 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2957109 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 2957109 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2957109 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2957109 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-01T06:09:56.446Z
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3605e46d{/solr,null,AVAILABLE}
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1669b1f5{SSL,[ssl, http/1.1]}{127.0.0.1:44654}
   [junit4]   2> 2957115 INFO  (jetty-launcher-3516-thread-2) [    ] o.e.j.s.Server Started @2957264ms
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44654}
   [junit4]   2> 2957116 ERROR (jetty-launcher-3516-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.4.0
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2957116 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-01T06:09:56.453Z
   [junit4]   2> 2957125 INFO  (zkConnectionManagerCallback-3523-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957125 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2957134 INFO  (zkConnectionManagerCallback-3521-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957134 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2957143 INFO  (jetty-launcher-3516-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50144/solr
   [junit4]   2> 2957152 INFO  (jetty-launcher-3516-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50144/solr
   [junit4]   2> 2957153 INFO  (zkConnectionManagerCallback-3527-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957155 INFO  (zkConnectionManagerCallback-3531-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957158 INFO  (zkConnectionManagerCallback-3533-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957170 INFO  (zkConnectionManagerCallback-3535-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957196 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2957197 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44654_solr
   [junit4]   2> 2957199 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.Overseer Overseer (id=73898191823175685-127.0.0.1:44654_solr-n_0000000000) starting
   [junit4]   2> 2957248 INFO  (zkConnectionManagerCallback-3544-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957250 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50144/solr ready
   [junit4]   2> 2957261 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2957262 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42455_solr
   [junit4]   2> 2957264 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2957265 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2957266 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 2957266 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2957294 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2957294 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44654_solr
   [junit4]   2> 2957307 INFO  (zkCallback-3543-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2957312 INFO  (OverseerStateUpdate-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2957322 INFO  (zkCallback-3543-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2957323 INFO  (zkCallback-3534-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2957336 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:42455_solr, 127.0.0.1:44654_solr]
   [junit4]   2> 2957342 INFO  (zkCallback-3534-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2957346 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2957346 DEBUG (ScheduledTrigger-5593-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2957346 DEBUG (OverseerAutoScalingTriggerThread-73898191823175685-127.0.0.1:44654_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2957707 WARN  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 2957803 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44654.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957823 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44654.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957823 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44654.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957826 INFO  (zkConnectionManagerCallback-3551-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957827 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2957828 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50144/solr ready
   [junit4]   2> 2957832 INFO  (jetty-launcher-3516-thread-2) [n:127.0.0.1:44654_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_4B37CA88DAB0C8C7-001/tempDir-001/node2/.
   [junit4]   2> 2957833 WARN  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 2957860 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42455.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957873 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42455.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957873 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42455.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e2753e
   [junit4]   2> 2957874 INFO  (jetty-launcher-3516-thread-1) [n:127.0.0.1:42455_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_4B37CA88DAB0C8C7-001/tempDir-001/node1/.
   [junit4]   2> 2957988 INFO  (zkConnectionManagerCallback-3553-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957991 INFO  (zkConnectionManagerCallback-3558-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2957992 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2957993 INFO  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50144/solr ready
   [junit4]   2> 2958036 DEBUG (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10004_solr
   [junit4]   2> 2958036 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2958037 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 2958037 WARN  (SUITE-IndexSizeTriggerTest-seed#[4B37CA88DAB0C8C7]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 2958040 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 2958040 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 2958043 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10005_solr]
   [junit4]   2> 2958050 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2958050 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 2958051 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958071 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958092 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958113 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958133 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958153 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958176 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958197 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958217 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958237 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958257 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958265 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[4B37CA88DAB0C8C7]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSplitIntegration
   [junit4]   2> 2958277 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958298 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958325 DEBUG (simCloudManagerPool-5605-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 2958325 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958335 DEBUG (simCloudManagerPool-5605-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 2958342 DEBUG (simCloudManagerPool-5605-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2958345 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958347 DEBUG (ScheduledTrigger-5593-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958348 DEBUG (simCloudManagerPool-5605-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2958348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2958349 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2958349 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2958366 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958379 DEBUG (simCloudManagerPool-5605-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2958379 DEBUG (simCloudManagerPool-5605-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 2958380 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 2958380 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2958380 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 2958383 DEBUG (simCloudManagerPool-5605-thread-10) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2958383 DEBUG (simCloudManagerPool-5605-thread-10) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 2958383 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 2958384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2958384 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 2958390 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958410 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958430 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958450 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958470 DEBUG (ScheduledTrigger-5606-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958491 DEBUG (ScheduledTrigger-5606-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958511 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958531 DEBUG (ScheduledTrigger-5606-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958552 DEBUG (ScheduledTrigger-5606-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 2958553 DEBUG (ScheduledTrigger-5606-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger, .auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 2958559 INFO  (ScheduledTrigger-5606-thread-3) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148265258184175, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958582 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 2958584 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148266480487125, stage=BEFORE_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958584 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 2958588 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 2958588 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28148266729 sessionWrapper.createTime 28148266569278675, this.sessionWrapper.createTime 28148266569278675 
   [junit4]   2> 2958589 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148266732998875, stage=AFTER_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958589 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148266758401325, stage=BEFORE_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958589 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@481366f3], AFTER_ACTION=[compute_plan], BEFORE_ACTION=[compute_plan, execute_plan]}
   [junit4]   2> 2958589 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 2958741 DEBUG (simCloudManagerPool-5605-thread-43) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_1: {"core_node10":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":7,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_1_replica1",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 2958746 DEBUG (simCloudManagerPool-5605-thread-44) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_0: {"core_node7":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10004/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":7,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 2958748 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148274701016725, stage=AFTER_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger/6400b12d68ec67T1tf3xi148osnwslk2g322wfgs/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger/6400b12d68ec67T1tf3xi148osnwslk2g322wfgs/0] in completed tasks},success=}], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958748 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148274728944425, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"6400b12d68ec67T1tf3xi148osnwslk2g322wfgs",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148258642127975,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148264964916175,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958751 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: index_size_trigger after 100ms
   [junit4]   2> 2958751 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 2958751 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 2958751 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 8427 ms for event id=6400b12d68ec67T1tf3xi148osnwslk2g322wfgs
   [junit4]   2> 2958851 DEBUG (ScheduledTrigger-5606-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2958852 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 2958852 DEBUG (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger, .auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 2958852 INFO  (ScheduledTrigger-5606-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148279915536575, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958853 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 2958853 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148279987172875, stage=BEFORE_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958854 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 2958855 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 2958855 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28148280049 sessionWrapper.createTime 28148280037498425, this.sessionWrapper.createTime 28148280037498425 
   [junit4]   2> 2958855 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148280051970775, stage=AFTER_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958855 INFO  (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=28148280068155725, stage=BEFORE_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 2958855 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"6400b16a537949T1tf3xi148osnwslk2g322wfgv",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":28148259664132425,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":1,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10004_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":28148279908113275,
   [junit4]   2>     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@1028da85], AFTER_ACTION=[compute_plan], BEFORE_ACTION=[compute_plan, execute_plan]}
   [junit4]   2> 2958856 DEBUG (AutoscalingActionExecutor-5607-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 2958951 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[4B37CA88DAB0C8C7]) [    ] o.a.s.c.a.IndexSizeTriggerTest #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[testSplitIntegration_collection, .system]
   [junit4]   2> DocCollection(testSplitIntegration_collection//clusterstate.json/41)={
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_replica_n3",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":11},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":11}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "stateTimestamp":"1527896376941472600",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":14},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":14}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"inactive"},
   [junit4]   2>     "shard1_1":{
   [junit4]   2>       "parent":"shard1",
   [junit4]   2>       "stateTimestamp":"1527896376976772250",
   [junit4]   2>       "range":"c0000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node10":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_1_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7},
   [junit4]   2>         "core_node9":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_1_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10004/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7}}},
   [junit4]   2>     "shard1_0":{
   [junit4]   2>       "parent":"shard1",
   [junit4]   2>       "stateTimestamp":"1527896376976435050",
   [junit4]   2>       "range":"80000000-bfffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node7":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_0_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10004/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7},
   [junit4]   2>         "core_node8":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_0_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7}}}}}
   [junit4]   2> DocCollection(.system//clusterstate.json/41)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node1":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":3}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> ## Max replicas per node:	5
   [junit4]   2> ## Min replicas per node:	4
   [junit4]   2> ## Total replicas:		9
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ## * testSplitIntegration_collection		8
   [junit4]   2> ##		- active           8
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- DELETESTATUS       1
   [junit4]   2> ##		- REQUESTSTATUS      1
   [junit4]   2> ##		- SPLITSHARD         2
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update 

[...truncated too long message...]

onfigure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1023667499
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1023667499
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1023667499

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null721303089
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null721303089
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null721303089

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 35ms :: artifacts dl 1ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 151 minutes 33 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-Tests-7.x - Build # 629 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/629/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded

Error Message:
ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5c255ba6, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@13dec136] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5c255ba6, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@13dec136] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([17A4214A9F9736A6:7267773D3D349EA5]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: DocCollection(testSplitIntegration_collection//clusterstate.json/71)={   "replicationFactor":"2",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testSplitIntegration_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11},         "core_node4":{           "core":"testSplitIntegration_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":11,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":11}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1527752717997368650",       "replicas":{         "core_node1":{           "core":"testSplitIntegration_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14},         "core_node2":{           "core":"testSplitIntegration_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":14,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":14}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1527752718033101400",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node9":{           "core":"testSplitIntegration_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1527752718032721150",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testSplitIntegration_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7},         "core_node8":{           "core":"testSplitIntegration_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":7,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":1,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "SEARCHER.searcher.numDocs":7}}}}}

Stack Trace:
java.util.concurrent.TimeoutException: last state: DocCollection(testSplitIntegration_collection//clusterstate.json/71)={
  "replicationFactor":"2",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testSplitIntegration_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11},
        "core_node4":{
          "core":"testSplitIntegration_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":11,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":11}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1527752717997368650",
      "replicas":{
        "core_node1":{
          "core":"testSplitIntegration_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14},
        "core_node2":{
          "core":"testSplitIntegration_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":14,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":14}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1527752718033101400",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node9":{
          "core":"testSplitIntegration_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1527752718032721150",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testSplitIntegration_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7},
        "core_node8":{
          "core":"testSplitIntegration_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":7,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":1,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "SEARCHER.searcher.numDocs":7}}}}}
	at __randomizedtesting.SeedInfo.seed([17A4214A9F9736A6:2E2A980AB068FF58]:0)
	at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
	at __randomizedtesting.SeedInfo.seed([17A4214A9F9736A6:441D63FA7D86A35C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:404)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13491 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction
   [junit4]   2> 1423029 INFO  (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestComputePlanAction_17A4214A9F9736A6-001/init-core-data-001
   [junit4]   2> 1423104 WARN  (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 1423105 INFO  (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1423106 INFO  (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1423185 DEBUG (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 1423187 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1423187 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1423188 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1423188 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1423194 WARN  (SUITE-TestComputePlanAction-seed#[17A4214A9F9736A6]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler Missing .system, keeping metrics history in memory
   [junit4]   2> 1423197 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
   [junit4]   2> 1423197 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1423198 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1423196 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 1423207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423207 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1423214 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1423215 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1423215 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1423215 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1423215 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1423215 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 1423216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1423216 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423237 DEBUG (ScheduledTrigger-3265-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423257 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423277 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423357 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423377 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423389 DEBUG (simCloudManagerPool-3264-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423390 DEBUG (simCloudManagerPool-3264-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1423390 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1423391 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423391 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1423398 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423405 DEBUG (simCloudManagerPool-3264-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423405 DEBUG (simCloudManagerPool-3264-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1423405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1423405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1423423 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423447 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423467 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423491 DEBUG (ScheduledTrigger-3265-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423508 DEBUG (simCloudManagerPool-3264-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423508 DEBUG (simCloudManagerPool-3264-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1423509 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1423509 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 1423509 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 1423509 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1423510 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423510 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423511 DEBUG (simCloudManagerPool-3264-thread-4) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423511 DEBUG (ScheduledTrigger-3265-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423511 DEBUG (simCloudManagerPool-3264-thread-4) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1423512 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1423512 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1423513 DEBUG (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
   [junit4]   2> 1423524 WARN  (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 1423529 DEBUG (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1423530 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423530 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423531 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423532 DEBUG (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1423540 INFO  (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003451845 sessionWrapper.createTime 28003451437917382, this.sessionWrapper.createTime 28003451437917382 
   [junit4]   2> 1423540 DEBUG (simCloudManagerPool-3264-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1423550 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423550 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423555 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423570 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423570 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423575 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423591 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423591 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423595 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423604 DEBUG (simCloudManagerPool-3264-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeAdded / shard1: {"core_node1":{
   [junit4]   2>     "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testNodeAdded"}}
   [junit4]   2> 1423611 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423611 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423616 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423625 DEBUG (simCloudManagerPool-3264-thread-11) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423625 DEBUG (simCloudManagerPool-3264-thread-11) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1423626 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1423626 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1423637 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423637 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 1423637 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423659 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423659 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1423659 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10001_solr at time 28003457798221432
   [junit4]   2> 1423659 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1423679 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1423679 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1423679 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10001_solr] added at times [28003457798221432], now=28003458819826082
   [junit4]   2> 1423679 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"637cff1a8bae78Taq61xtltuxi6fmj0j9gut5tt1",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":28003457798221432,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003457798221432],
   [junit4]   2>     "nodeNames":["127.0.0.1:10001_solr"]}}
   [junit4]   2> 1423680 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_added_trigger]
   [junit4]   2> 1423682 DEBUG (simCloudManagerPool-3264-thread-12) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 1423693 DEBUG (simCloudManagerPool-3264-thread-12) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1423693 DEBUG (simCloudManagerPool-3264-thread-12) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1423694 INFO  (simCloudManagerPool-3264-thread-12) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003459571 sessionWrapper.createTime 28003459519218832, this.sessionWrapper.createTime 28003459519218832 
   [junit4]   2> 1423694 DEBUG (simCloudManagerPool-3264-thread-12) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1423709 WARN  (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: org.apache.solr.common.SolrException: Collection .system doesn't exist.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:621) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:790) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:757) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$4(ScheduledTriggers.java:283) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.NodeAddedTrigger.run(NodeAddedTrigger.java:161) [java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:589) [java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: org.apache.solr.common.SolrException: Collection .system doesn't exist.
   [junit4]   2> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:619) ~[test/:?]
   [junit4]   2> 	... 13 more
   [junit4]   2> Caused by: java.io.IOException: org.apache.solr.common.SolrException: Collection .system doesn't exist.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simUpdate(SimClusterStateProvider.java:1201) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:691) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:618) ~[test/:?]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	... 3 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Collection .system doesn't exist.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simSetShardValue(SimClusterStateProvider.java:1392) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simUpdate(SimClusterStateProvider.java:1195) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:691) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$15(SimCloudManager.java:618) ~[test/:?]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	... 3 more
   [junit4]   2> 1423716 DEBUG (ScheduledTrigger-3265-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10001_solr
   [junit4]   2> 1423716 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"637cff1a8bae78Taq61xtltuxi6fmj0j9gut5tt1",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":28003457798221432,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003457798221432],
   [junit4]   2>     "_enqueue_time_":28003458853938582,
   [junit4]   2>     "nodeNames":["127.0.0.1:10001_solr"]}}
   [junit4]   2> 1423717 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"637cff1a8bae78Taq61xtltuxi6fmj0j9gut5tt1",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":28003457798221432,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003457798221432],
   [junit4]   2>     "_enqueue_time_":28003458853938582,
   [junit4]   2>     "nodeNames":["127.0.0.1:10001_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 1423717 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1423718 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1423723 INFO  (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10001_solr&inPlaceMove=true&replica=core_node1
   [junit4]   2> 1423724 INFO  (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10001_solr&inPlaceMove=true&replica=core_node2
   [junit4]   2> 1423725 INFO  (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003461120 sessionWrapper.createTime 28003460994908232, this.sessionWrapper.createTime 28003460994908232 
   [junit4]   2> 1423725 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1423727 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction Node values: {
   [junit4]   2>   "127.0.0.1:10001_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10001_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10001,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10000_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":4,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":996,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10000_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10000,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 1423728 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction Live nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr], collection state: DocCollection(testNodeAdded//clusterstate.json/8)={
   [junit4]   2>   "replicationFactor":"4",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"4",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 1423728 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
   [junit4]   2> 1423728 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10001_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10001_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10001,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10000_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":4,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":996,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10000_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10000,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 1423728 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1423729 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/9)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node1":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":2,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 1423729 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeAdded state: DocCollection(testNodeAdded//clusterstate.json/9)={
   [junit4]   2>   "replicationFactor":"4",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"4",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 1423730 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAdded
   [junit4]   2> 1423737 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		2
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		0
   [junit4]   2> ## Collections:	[.system, testNodeAdded]
   [junit4]   2> ## Max replicas per node:	4
   [junit4]   2> ## Min replicas per node:	1
   [junit4]   2> ## Total replicas:		5
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ## * testNodeAdded		4
   [junit4]   2> ##		- active           4
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- autoscaling        5
   [junit4]   2> ##		- update             3
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger
   [junit4]   2> ##		- AFTER_ACTION     1
   [junit4]   2> ##		- BEFORE_ACTION     1
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestComputePlanAction -Dtests.method=testNodeAdded -Dtests.seed=17A4214A9F9736A6 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-US -Dtests.timezone=Asia/Ashkhabad -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.55s J0 | TestComputePlanAction.testNodeAdded <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5c255ba6, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@13dec136] expected:<1> but was:<2>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([17A4214A9F9736A6:7267773D3D349EA5]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1423751 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1423751 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 1423751 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1423751 DEBUG (AutoscalingActionExecutor-3266-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1731 ms for event id=637cff1a8bae78Taq61xtltuxi6fmj0j9gut5tt1
   [junit4]   2> 1423759 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
   [junit4]   2> 1423759 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 1423759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 1423760 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1423760 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423760 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 1423761 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10001_solr
   [junit4]   2> 1423779 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1423779 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1423779 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1423784 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1423784 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 1423785 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
   [junit4]   2> 1423785 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 1423790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr]
   [junit4]   2> 1423790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423790 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 1423791 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423811 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423835 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423859 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423879 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423882 DEBUG (simCloudManagerPool-3267-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423882 DEBUG (simCloudManagerPool-3267-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 1423883 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 1423883 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423883 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 1423883 DEBUG (simCloudManagerPool-3267-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423884 DEBUG (simCloudManagerPool-3267-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 1423886 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 1423886 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423886 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 1423900 DEBUG (ScheduledTrigger-3268-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423920 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423940 DEBUG (ScheduledTrigger-3268-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423961 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423981 DEBUG (ScheduledTrigger-3268-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1423993 DEBUG (simCloudManagerPool-3267-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1423994 DEBUG (simCloudManagerPool-3267-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 1423995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 1423995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 1423995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1423995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 1423995 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424003 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424003 DEBUG (simCloudManagerPool-3267-thread-4) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeWithMultipleReplicasLost
   [junit4]   2> 1424016 DEBUG (simCloudManagerPool-3267-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1424017 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424027 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424027 DEBUG (simCloudManagerPool-3267-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1424041 DEBUG (ScheduledTrigger-3268-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424051 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424060 INFO  (simCloudManagerPool-3267-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003477833 sessionWrapper.createTime 28003476199701882, this.sessionWrapper.createTime 28003476199701882 
   [junit4]   2> 1424060 DEBUG (simCloudManagerPool-3267-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1424061 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424071 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424081 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424091 DEBUG (ScheduledTrigger-3268-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424101 DEBUG (ScheduledTrigger-3268-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424111 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1424117 DEBUG (simCloudManagerPool-3267-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard2: {"core_node4":{
   [junit4]   2>     "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testNodeWithMultipleReplicasLost"}}
   [junit4]   2> 1424119 DEBUG (simCloudManagerPool-3267-thread-12) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard1: {"core_node1":{
   [junit4]   2>     "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testNodeWithMultipleReplicasLost"}}
   [junit4]   2> 1424122 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 1424122 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 15
   [junit4]   2> live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> collections:{testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/14)={
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}}
   [junit4]   2> 1424131 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1424131 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
   [junit4]   2> 1424142 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 1424142 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
   [junit4]   2> 1424152 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1424162 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 1424162 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10004_solr]
   [junit4]   2> 1424162 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"637d04b9b2748cTaq61xtltuxi6fmj0j9gut5tt6",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":28003481943176332,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003481943176332],
   [junit4]   2>     "nodeNames":["127.0.0.1:10004_solr"]}}
   [junit4]   2> 1424162 DEBUG (ScheduledTrigger-3268-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 1424164 DEBUG (simCloudManagerPool-3267-thread-15) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 1424179 DEBUG (simCloudManagerPool-3267-thread-15) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1424180 DEBUG (simCloudManagerPool-3267-thread-15) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1424181 INFO  (simCloudManagerPool-3267-thread-15) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003483923 sessionWrapper.createTime 28003483849516032, this.sessionWrapper.createTime 28003483849516032 
   [junit4]   2> 1424181 DEBUG (simCloudManagerPool-3267-thread-15) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1424193 DEBUG (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"637d04b9b2748cTaq61xtltuxi6fmj0j9gut5tt6",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":28003481943176332,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003481943176332],
   [junit4]   2>     "_enqueue_time_":28003482975673682,
   [junit4]   2>     "nodeNames":["127.0.0.1:10004_solr"]}}
   [junit4]   2> 1424193 DEBUG (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"637d04b9b2748cTaq61xtltuxi6fmj0j9gut5tt6",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":28003481943176332,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[28003481943176332],
   [junit4]   2>     "_enqueue_time_":28003482975673682,
   [junit4]   2>     "nodeNames":["127.0.0.1:10004_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 1424193 DEBUG (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 1424201 DEBUG (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 1424207 INFO  (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
   [junit4]   2> 1424209 INFO  (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10001_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 1424212 INFO  (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 28003485449 sessionWrapper.createTime 28003484914544282, this.sessionWrapper.createTime 28003484914544282 
   [junit4]   2> 1424212 DEBUG (AutoscalingActionExecutor-3269-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 1424213 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
   [junit4]   2> 1424213 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10001_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 1424214 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
   [junit4]   2> 1424214 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10001_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10001_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10001,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10004_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":998,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10004_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10004,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10002_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":998,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10002_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10002,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10003_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":998,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10003,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 1424214 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 1424215 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/21)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node2":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":2,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 1424215 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[17A4214A9F9736A6]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeWithMultipleReplicasLost state: DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/21)={
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "lead

[...truncated too long message...]

onfigure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1686458411
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1686458411
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null1686458411

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null499220915
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null499220915
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null499220915

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 40ms :: artifacts dl 2ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 146 minutes 34 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)