You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/06/19 21:46:22 UTC

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-13.0.2) - Build # 6186 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6186/
Java: 64bit/jdk-13.0.2 -XX:+UseCompressedOops -XX:+UseSerialGC

5 tests failed.
FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
KeeperErrorCode = Session expired for /overseer_elect/leader

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
	at __randomizedtesting.SeedInfo.seed([68DCC9A71CA3D949:E088F67DB25FB4B1]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131)
	at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
	at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderId(OverseerTaskProcessor.java:420)
	at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderNode(OverseerTaskProcessor.java:411)
	at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:138)
	at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:79)
	at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:53)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:830)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest:     1) Thread[id=3223, name=SessionTracker, state=TIMED_WAITING, group=TGRP-JsonRequestApiHeatmapFacetingTest]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest: 
   1) Thread[id=3223, name=SessionTracker, state=TIMED_WAITING, group=TGRP-JsonRequestApiHeatmapFacetingTest]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([7735949F21616FB3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.ref_guide_examples.JsonRequestApiTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.JsonRequestApiTest:     1) Thread[id=3315, name=SessionTracker, state=TIMED_WAITING, group=TGRP-JsonRequestApiTest]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.JsonRequestApiTest: 
   1) Thread[id=3315, name=SessionTracker, state=TIMED_WAITING, group=TGRP-JsonRequestApiTest]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([7735949F21616FB3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest:     1) Thread[id=3405, name=SessionTracker, state=TIMED_WAITING, group=TGRP-UsingSolrJRefGuideExamplesTest]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest: 
   1) Thread[id=3405, name=SessionTracker, state=TIMED_WAITING, group=TGRP-UsingSolrJRefGuideExamplesTest]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([7735949F21616FB3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.common.cloud.TestDocCollectionWatcher

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.common.cloud.TestDocCollectionWatcher:     1) Thread[id=1395, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestDocCollectionWatcher]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.common.cloud.TestDocCollectionWatcher: 
   1) Thread[id=1395, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestDocCollectionWatcher]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([7735949F21616FB3]:0)




Build Log:
[...truncated 13754 lines...]
   [junit4] Suite: org.apache.solr.cloud.RollingRestartTest
   [junit4]   2> 1164408 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1164408 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1164409 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/data-dir-126-001
   [junit4]   2> 1164409 WARN  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 1164409 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1164410 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1164410 INFO  (SUITE-RollingRestartTest-seed#[68DCC9A71CA3D949]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ui_zk/lq
   [junit4]   2> 1164412 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1164413 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1164413 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1164518 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer start zk server on port: 54289
   [junit4]   2> 1164518 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:54289
   [junit4]   2> 1164518 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:54289
   [junit4]   2> 1164518 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 54289
   [junit4]   2> 1164520 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1164523 INFO  (zkConnectionManagerCallback-10348-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1164523 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1164525 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1164528 INFO  (zkConnectionManagerCallback-10350-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1164528 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1164529 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1164531 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1164533 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1164535 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1164537 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1164539 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1164540 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1164542 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1164544 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1164545 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1164547 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1164548 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 1164864 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1164864 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1164864 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 13.0.2+8
   [junit4]   2> 1164866 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1164866 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1164866 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1164867 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45dcac4c{/ui_zk/lq,null,AVAILABLE}
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@66a69cfc{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:54294}
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.s.Server Started @1164918ms
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ui_zk/lq, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/tempDir-001/control/data, hostPort=54294, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/control-001/cores, replicaType=NRT}
   [junit4]   2> 1164868 ERROR (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-19T21:11:38.121552Z
   [junit4]   2> 1164868 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1164874 INFO  (zkConnectionManagerCallback-10352-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1164874 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1164981 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1164981 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/control-001/solr.xml
   [junit4]   2> 1164983 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1165066 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1165067 WARN  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4b0ffd3a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1165067 WARN  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4b0ffd3a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1165069 WARN  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2acca1a8[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1165069 WARN  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2acca1a8[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1165069 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54289/solr
   [junit4]   2> 1165070 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1165073 INFO  (zkConnectionManagerCallback-10363-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1165073 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1165185 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1165188 INFO  (zkConnectionManagerCallback-10365-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1165188 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1165329 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54294_ui_zk%2Flq
   [junit4]   2> 1165330 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.Overseer Overseer (id=72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) starting
   [junit4]   2> 1165340 INFO  (OverseerStateUpdate-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:54294_ui_zk%2Flq
   [junit4]   2> 1165340 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54294_ui_zk%2Flq
   [junit4]   2> 1165342 INFO  (OverseerStateUpdate-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1165346 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1165346 WARN  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1165362 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1165379 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1165387 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1165387 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1165388 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/control-001/cores
   [junit4]   2> 1165393 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1165395 INFO  (zkConnectionManagerCallback-10382-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1165395 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1165397 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1165398 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54289/solr ready
   [junit4]   2> 1165400 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:54294_ui_zk%252Flq&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1165406 INFO  (OverseerThreadFactory-10372-thread-1-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1165524 INFO  (qtp2107574328-15972) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1165524 INFO  (qtp2107574328-15972) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1165531 INFO  (OverseerStateUpdate-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"control_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:54294/ui_zk/lq",
   [junit4]   2>   "node_name":"127.0.0.1:54294_ui_zk%2Flq",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1165748 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq    x:control_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1165759 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1165769 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1165839 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1165862 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1165862 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1165862 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1165865 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1165903 WARN  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}}
   [junit4]   2> 1165945 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1165945 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1165946 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1165946 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1165947 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1165949 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1165949 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1165950 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1165950 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669963290197884928
   [junit4]   2> 1165952 INFO  (searcherExecutor-10384-thread-1-processing-n:127.0.0.1:54294_ui_zk%2Flq x:control_collection_shard1_replica_n1 c:control_collection s:shard1 r:core_node2) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1165956 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 1165956 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 1165962 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1165962 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1165962 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54294/ui_zk/lq/control_collection_shard1_replica_n1/
   [junit4]   2> 1165962 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1165963 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:54294/ui_zk/lq/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1165963 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72133566541332484-core_node2-n_0000000000
   [junit4]   2> 1165965 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54294/ui_zk/lq/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1166072 INFO  (zkCallback-10364-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1166072 INFO  (zkCallback-10364-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1166074 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1166076 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=328
   [junit4]   2> 1166079 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1166179 INFO  (zkCallback-10364-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1166179 INFO  (zkCallback-10364-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1166179 INFO  (zkCallback-10364-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1166180 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:54294_ui_zk%252Flq&wt=javabin&version=2} status=0 QTime=780
   [junit4]   2> 1166181 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection
   [junit4]   2> 1166294 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166296 INFO  (zkConnectionManagerCallback-10393-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1166297 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1166298 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1166300 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54289/solr ready
   [junit4]   2> 1166300 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1166301 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1166308 INFO  (OverseerCollectionConfigSetProcessor-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1166308 INFO  (OverseerThreadFactory-10372-thread-2-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1166518 WARN  (OverseerThreadFactory-10372-thread-2-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1166520 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1166522 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&wt=javabin&version=2} status=0 QTime=220
   [junit4]   2> 1166523 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 1166523 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1166524 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 1166524 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1166524 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 1166524 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1166525 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=2
   [junit4]   2> 1166899 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001 of type NRT for shard2
   [junit4]   2> 1166900 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1166900 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1166900 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 13.0.2+8
   [junit4]   2> 1166902 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1166902 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1166902 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1166902 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1bd70330{/ui_zk/lq,null,AVAILABLE}
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@12b3ddc1{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:54303}
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.e.j.s.Server Started @1166953ms
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ui_zk/lq, solrconfig=solrconfig.xml, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/tempDir-001/jetty1, hostPort=54303, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001/cores}
   [junit4]   2> 1166903 ERROR (closeThreadPool-10394-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1166903 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-19T21:11:40.156914Z
   [junit4]   2> 1166904 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1166907 INFO  (zkConnectionManagerCallback-10396-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1166907 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167015 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1167015 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001/solr.xml
   [junit4]   2> 1167018 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1167057 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1167057 WARN  (closeThreadPool-10394-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2c681c56[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167057 WARN  (closeThreadPool-10394-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2c681c56[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167059 WARN  (closeThreadPool-10394-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@74f35eba[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167059 WARN  (closeThreadPool-10394-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@74f35eba[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167060 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54289/solr
   [junit4]   2> 1167061 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1167064 INFO  (zkConnectionManagerCallback-10407-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1167064 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167168 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1167170 INFO  (zkConnectionManagerCallback-10409-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1167170 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167180 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1167188 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.ZkController Publish node=127.0.0.1:54303_ui_zk%2Flq as DOWN
   [junit4]   2> 1167190 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1167190 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54303_ui_zk%2Flq
   [junit4]   2> 1167193 INFO  (zkCallback-10364-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1167193 INFO  (zkCallback-10392-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1167194 INFO  (zkCallback-10408-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1167197 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1167197 WARN  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1167236 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001 of type NRT for shard1
   [junit4]   2> 1167237 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1167237 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1167237 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 13.0.2+8
   [junit4]   2> 1167238 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1167238 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1167238 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1167238 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d60e456{/ui_zk/lq,null,AVAILABLE}
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@363c93d0{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:54307}
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.e.j.s.Server Started @1167289ms
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ui_zk/lq, solrconfig=solrconfig.xml, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/tempDir-001/jetty2, hostPort=54307, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001/cores}
   [junit4]   2> 1167239 ERROR (closeThreadPool-10394-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1167239 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-19T21:11:40.492721Z
   [junit4]   2> 1167240 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1167242 INFO  (zkConnectionManagerCallback-10415-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1167242 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167245 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1167262 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167270 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167270 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167271 INFO  (closeThreadPool-10394-thread-1) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001/cores
   [junit4]   2> 1167276 INFO  (closeThreadPool-10394-thread-1) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:54303_ui_zk%2Flq
   [junit4]   2> 1167344 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1167344 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001/solr.xml
   [junit4]   2> 1167347 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1167424 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1167425 WARN  (closeThreadPool-10394-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4cfa9e1a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167425 WARN  (closeThreadPool-10394-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4cfa9e1a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167428 WARN  (closeThreadPool-10394-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@28074826[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167428 WARN  (closeThreadPool-10394-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@28074826[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1167429 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54289/solr
   [junit4]   2> 1167430 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1167433 INFO  (zkConnectionManagerCallback-10430-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1167433 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167547 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1167550 INFO  (zkConnectionManagerCallback-10432-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1167550 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1167570 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1167582 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.ZkController Publish node=127.0.0.1:54307_ui_zk%2Flq as DOWN
   [junit4]   2> 1167584 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1167584 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54307_ui_zk%2Flq
   [junit4]   2> 1167587 INFO  (zkCallback-10364-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1167587 INFO  (zkCallback-10408-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1167587 INFO  (zkCallback-10392-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1167589 INFO  (zkCallback-10431-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1167595 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1167597 WARN  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1167623 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1167648 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167659 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167660 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1167661 INFO  (closeThreadPool-10394-thread-2) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001/cores
   [junit4]   2> 1167669 INFO  (closeThreadPool-10394-thread-2) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:54307_ui_zk%2Flq
   [junit4]   2> 1167673 INFO  (qtp2107574328-15971) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:54303_ui_zk%252Flq&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1167674 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:54307_ui_zk%252Flq&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1167688 INFO  (OverseerCollectionConfigSetProcessor-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1167700 INFO  (qtp2107574328-15970) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=3
   [junit4]   2> 1167702 INFO  (qtp2107574328-15972) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1167707 INFO  (qtp378293192-16027) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1167709 INFO  (qtp839057736-16052) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1167713 INFO  (OverseerThreadFactory-10372-thread-4-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:54307_ui_zk%2Flq for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 1167721 INFO  (OverseerThreadFactory-10372-thread-3-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1 s:shard2   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:54303_ui_zk%2Flq for creating new replica of shard shard2 for collection collection1
   [junit4]   2> 1167723 INFO  (OverseerThreadFactory-10372-thread-4-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 1167729 INFO  (OverseerThreadFactory-10372-thread-3-processing-n:127.0.0.1:54294_ui_zk%2Flq) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1 s:shard2   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 1167729 INFO  (OverseerStateUpdate-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"addreplica",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:54307/ui_zk/lq",
   [junit4]   2>   "node_name":"127.0.0.1:54307_ui_zk%2Flq",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 1167735 INFO  (OverseerStateUpdate-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"addreplica",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:54303/ui_zk/lq",
   [junit4]   2>   "node_name":"127.0.0.1:54303_ui_zk%2Flq",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 1167943 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq    x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1167951 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq    x:collection1_shard2_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=collection1_shard2_replica_n2&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1167986 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1167988 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1168001 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1168025 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1168157 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1168222 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1168283 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from configset conf1, trusted=true
   [junit4]   2> 1168284 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1168284 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001/cores/collection1_shard2_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-1-001/cores/collection1_shard2_replica_n2/data/]
   [junit4]   2> 1168292 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1168308 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1168309 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@524b9034
   [junit4]   2> 1168310 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001/cores/collection1_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.RollingRestartTest_68DCC9A71CA3D949-001/shard-2-001/cores/collection1_shard1_replica_n1/data/]
   [junit4]   2> 1168324 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1168326 WARN  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}}
   [junit4]   2> 1168344 WARN  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}}
   [junit4]   2> 1168472 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1168472 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1168474 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1168474 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1168476 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1168480 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1168480 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1168482 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1168482 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1168482 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1168483 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1168484 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1168487 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1168488 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669963292859170816
   [junit4]   2> 1168491 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1168493 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1168493 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1168494 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669963292865462272
   [junit4]   2> 1168497 INFO  (searcherExecutor-10445-thread-1-processing-n:127.0.0.1:54307_ui_zk%2Flq x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1168502 INFO  (searcherExecutor-10443-thread-1-processing-n:127.0.0.1:54303_ui_zk%2Flq x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrCore [collection1_shard2_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1168514 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1168514 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
   [junit4]   2> 1168515 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 1168515 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 1168529 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1168529 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1168529 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1168529 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54303/ui_zk/lq/collection1_shard2_replica_n2/
   [junit4]   2> 1168529 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1168529 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54307/ui_zk/lq/collection1_shard1_replica_n1/
   [junit4]   2> 1168530 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1168530 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1168531 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy http://127.0.0.1:54303/ui_zk/lq/collection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 1168531 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72133566541332489-core_node4-n_0000000000
   [junit4]   2> 1168531 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:54307/ui_zk/lq/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 1168531 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72133566541332492-core_node3-n_0000000000
   [junit4]   2> 1168537 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54307/ui_zk/lq/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 1168538 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54303/ui_zk/lq/collection1_shard2_replica_n2/ shard2
   [junit4]   2> 1168645 INFO  (zkCallback-10431-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168646 INFO  (zkCallback-10408-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168646 INFO  (zkCallback-10408-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168646 INFO  (zkCallback-10431-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168648 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1168653 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1168661 INFO  (qtp839057736-16051) [n:127.0.0.1:54307_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=718
   [junit4]   2> 1168666 INFO  (qtp378293192-16026) [n:127.0.0.1:54303_ui_zk%2Flq     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=collection1_shard2_replica_n2&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=715
   [junit4]   2> 1168670 INFO  (qtp2107574328-15969) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:54307_ui_zk%252Flq&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=996
   [junit4]   2> 1168673 INFO  (qtp2107574328-15971) [n:127.0.0.1:54294_ui_zk%2Flq c:collection1    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:54303_ui_zk%252Flq&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1000
   [junit4]   2> 1168673 INFO  (TEST-RollingRestartTest.test-seed#[68DCC9A71CA3D949]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 2 active replicas in collection: collection1
   [junit4]   2> 1168674 INFO  (OverseerCollectionConfigSetProcessor-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1168677 INFO  (OverseerCollectionConfigSetProcessor-72133566541332484-127.0.0.1:54294_ui_zk%2Flq-n_0000000000) [n:127.0.0.1:54294_ui_zk%2Flq     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000005 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1168760 INFO  (zkCallback-10408-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168760 INFO  (zkCallback-10392-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168760 INFO  (zkCallback-10408-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168760 INFO  (zkCallback-10431-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168760 INFO  (zkCallback-10431-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168760 INFO  (zkCallback-10408-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1168761 INFO  (zkCallback-10431-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has oc

[...truncated too long message...]

5154 INFO  (closeThreadPool-2435-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72133626921353220-127.0.0.1:62548_solr-n_0000000000) closing
   [junit4]   2> 175154 INFO  (OverseerStateUpdate-72133626921353220-127.0.0.1:62548_solr-n_0000000000) [n:127.0.0.1:62548_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:62548_solr
   [junit4]   2> 175155 INFO  (OverseerAutoScalingTriggerThread-72133626921353220-127.0.0.1:62548_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 175159 INFO  (closeThreadPool-2435-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72133626921353220-127.0.0.1:62548_solr-n_0000000000) closing
   [junit4]   2> 175265 INFO  (jetty-closer-2429-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72133626921353220-127.0.0.1:62548_solr-n_0000000000) closing
   [junit4]   2> 175266 INFO  (jetty-closer-2429-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@5a63f1aa{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 175267 INFO  (jetty-closer-2429-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@50f09c9f{/solr,null,UNAVAILABLE}
   [junit4]   2> 175267 INFO  (jetty-closer-2429-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 175268 INFO  (SUITE-UsingSolrJRefGuideExamplesTest-seed#[7735949F21616FB3]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 175494 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/collections/techproducts/terms/shard1
   [junit4]   2> 	2	/solr/aliases.json
   [junit4]   2> 	2	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	10	/solr/collections/techproducts/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/overseer/queue
   [junit4]   2> 	4	/solr/collections
   [junit4]   2> 	4	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 175500 INFO  (SUITE-UsingSolrJRefGuideExamplesTest-seed#[7735949F21616FB3]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:62542
   [junit4]   2> 175500 INFO  (SUITE-UsingSolrJRefGuideExamplesTest-seed#[7735949F21616FB3]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:62542
   [junit4]   2> 175500 INFO  (SUITE-UsingSolrJRefGuideExamplesTest-seed#[7735949F21616FB3]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 62542
   [junit4]   2> 175500 INFO  (SUITE-UsingSolrJRefGuideExamplesTest-seed#[7735949F21616FB3]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Jun 19, 2020 9:27:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Jun 19, 2020 9:27:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest: 
   [junit4]   2>    1) Thread[id=3405, name=SessionTracker, state=TIMED_WAITING, group=TGRP-UsingSolrJRefGuideExamplesTest]
   [junit4]   2>         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]   2> Jun 19, 2020 9:27:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=3405, name=SessionTracker, state=TIMED_WAITING, group=TGRP-UsingSolrJRefGuideExamplesTest]
   [junit4]   2> 185600 ERROR (SessionTracker) [     ] o.a.z.s.ZooKeeperCriticalThread Severe unrecoverable error, from thread : SessionTracker
   [junit4]   2>           => java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.base/java.lang.Thread.sleep(Native Method)
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method) [?:?]
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) ~[zookeeper-3.5.7.jar:3.5.7]
   [junit4]   2> Jun 19, 2020 9:27:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {}, docValues:{}, maxPointsInLeafNode=870, maxMBSortInHeap=5.115826252045321, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7f5fc294), locale=de, timezone=America/Argentina/Buenos_Aires
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 13.0.2 (64-bit)/cpus=6,threads=1,free=31807032,total=194707456
   [junit4]   2> NOTE: All tests run in this JVM: [SolrExampleBinaryHttp2Test, TestDocumentObjectBinder, CubedRootEvaluatorTest, CloudSolrClientMultiConstructorTest, ConcurrentUpdateHttp2SolrClientTest, TestV2Request, TestJSONWriter, TestZkConfigManager, FieldAnalysisResponseTest, LargeVolumeJettyTest, TestSolrProperties, CeilingEvaluatorTest, SolrExampleStreamingBinaryTest, SelectWithEvaluatorsTest, SchemaTest, TestValidatingJsonMap, Http2SolrClientCompatibilityTest, NormalizeEvaluatorTest, RecursiveEvaluatorTest, ExclusiveOrEvaluatorTest, JdbcDriverTest, JsonRequestApiHeatmapFacetingTest, JsonRequestApiTest, UsingSolrJRefGuideExamplesTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UsingSolrJRefGuideExamplesTest -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=de -Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | UsingSolrJRefGuideExamplesTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest: 
   [junit4]    >    1) Thread[id=3405, name=SessionTracker, state=TIMED_WAITING, group=TGRP-UsingSolrJRefGuideExamplesTest]
   [junit4]    >         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]    >         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7735949F21616FB3]:0)
   [junit4] Completed [89/208 (4!)] on J1 in 12.94s, 6 tests, 1 error <<< FAILURES!

[...truncated 51247 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6186/consoleText

[repro] Revision: 972c84022ec9c2b867848cb11f6d4c6bc466e8f2

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC"
[repro] Repro line:  ant test  -Dtestcase=RollingRestartTest -Dtests.method=test -Dtests.seed=68DCC9A71CA3D949 -Dtests.slow=true -Dtests.locale=os -Dtests.timezone=Turkey -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestDocCollectionWatcher -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=km -Dtests.timezone=Asia/Rangoon -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=JsonRequestApiHeatmapFacetingTest -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=fo -Dtests.timezone=Japan -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=JsonRequestApiTest -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=no-NO -Dtests.timezone=Asia/Ust-Nera -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=UsingSolrJRefGuideExamplesTest -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=de -Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       RollingRestartTest
[repro]    solr/solrj
[repro]       UsingSolrJRefGuideExamplesTest
[repro]       JsonRequestApiHeatmapFacetingTest
[repro]       JsonRequestApiTest
[repro]       TestDocCollectionWatcher
[repro] ant compile-test

[...truncated 2471 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.RollingRestartTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=68DCC9A71CA3D949 -Dtests.slow=true -Dtests.locale=os -Dtests.timezone=Turkey -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 86 lines...]
[repro] ant compile-test

[...truncated 451 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=20 -Dtests.class="*.UsingSolrJRefGuideExamplesTest|*.JsonRequestApiHeatmapFacetingTest|*.JsonRequestApiTest|*.TestDocCollectionWatcher" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=7735949F21616FB3 -Dtests.slow=true -Dtests.locale=de -Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 301 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest
[repro]   0/5 failed: org.apache.solr.client.ref_guide_examples.JsonRequestApiTest
[repro]   0/5 failed: org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest
[repro]   0/5 failed: org.apache.solr.cloud.RollingRestartTest
[repro]   0/5 failed: org.apache.solr.common.cloud.TestDocCollectionWatcher
[repro] Exiting with code 0

[...truncated 73 lines...]

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-14.0.1) - Build # 6188 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6188/
Java: 64bit/jdk-14.0.1 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ExecutePlanActionTest.testTaskTimeout

Error Message:
did not finish processing event in time

Stack Trace:
java.lang.AssertionError: did not finish processing event in time
	at __randomizedtesting.SeedInfo.seed([4336E26B99CC21FE:466BDA8104A4E369]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.autoscaling.ExecutePlanActionTest.testTaskTimeout(ExecutePlanActionTest.java:317)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest.testCooldown

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([4336E26B99CC21FE:72888F8FE766540C]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest.testCooldown(TriggerCooldownIntegrationTest.java:155)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)




Build Log:
[...truncated 13420 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest
   [junit4]   2> 1183284 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1183284 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1183285 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/data-dir-77-001
   [junit4]   2> 1183285 WARN  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1183285 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1183286 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1183286 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001
   [junit4]   2> 1183286 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1183286 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1183287 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1183394 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.ZkTestServer start zk server on port: 49980
   [junit4]   2> 1183394 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:49980
   [junit4]   2> 1183394 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:49980
   [junit4]   2> 1183394 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 49980
   [junit4]   2> 1183396 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183409 INFO  (zkConnectionManagerCallback-9023-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183409 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183413 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183416 INFO  (zkConnectionManagerCallback-9025-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183416 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183417 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183420 INFO  (zkConnectionManagerCallback-9027-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183421 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1183526 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1183528 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1183528 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1183528 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cac2219{/solr,null,AVAILABLE}
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35da0cfe{/solr,null,AVAILABLE}
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@74643f89{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:49987}
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.e.j.s.Server Started @1183607ms
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49987}
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@2151ebbe{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:49988}
   [junit4]   2> 1183529 ERROR (jetty-launcher-9028-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.e.j.s.Server Started @1183607ms
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49988}
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1183529 ERROR (jetty-launcher-9028-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T06:54:42.456369Z
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T06:54:42.456369Z
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183529 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183536 INFO  (zkConnectionManagerCallback-9030-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183536 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183536 INFO  (zkConnectionManagerCallback-9032-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183536 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183537 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1183537 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1183646 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1183647 WARN  (jetty-launcher-9028-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54c9543c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183647 WARN  (jetty-launcher-9028-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54c9543c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183649 WARN  (jetty-launcher-9028-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@11f0ba4c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183649 WARN  (jetty-launcher-9028-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@11f0ba4c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183650 INFO  (jetty-launcher-9028-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49980/solr
   [junit4]   2> 1183650 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183653 INFO  (zkConnectionManagerCallback-9043-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183653 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183754 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1183755 WARN  (jetty-launcher-9028-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@748ccaf3[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183755 WARN  (jetty-launcher-9028-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@748ccaf3[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183757 WARN  (jetty-launcher-9028-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@368b8bb6[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183757 WARN  (jetty-launcher-9028-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@368b8bb6[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1183758 INFO  (jetty-launcher-9028-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49980/solr
   [junit4]   2> 1183759 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183764 INFO  (zkConnectionManagerCallback-9056-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183764 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183796 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49988_solr
   [junit4]   2> 1183797 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.Overseer Overseer (id=72135859289063430-127.0.0.1:49988_solr-n_0000000000) starting
   [junit4]   2> 1183810 INFO  (OverseerStateUpdate-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [n:127.0.0.1:49988_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:49988_solr
   [junit4]   2> 1183810 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49988_solr
   [junit4]   2> 1183811 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1183813 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1183814 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 0
   [junit4]   2> 1183814 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1183814 INFO  (OverseerStateUpdate-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [n:127.0.0.1:49988_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1183819 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1183819 WARN  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1183824 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:49988_solr]
   [junit4]   2> 1183825 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 1183827 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 1183827 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1183840 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1183851 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1183856 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:49987_solr as DOWN
   [junit4]   2> 1183858 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1183858 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49987_solr
   [junit4]   2> 1183861 INFO  (zkCallback-9055-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1183861 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1183863 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1183864 WARN  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1183866 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49988.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183874 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49988.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183874 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49988.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183875 INFO  (jetty-launcher-9028-thread-1) [n:127.0.0.1:49988_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001/node1
   [junit4]   2> 1183879 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1183906 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49987.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183919 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49987.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183919 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49987.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1183920 INFO  (jetty-launcher-9028-thread-2) [n:127.0.0.1:49987_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001/node2
   [junit4]   2> 1183926 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
   [junit4]   2> 1183928 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1183930 INFO  (zkConnectionManagerCallback-9079-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1183930 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1183936 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1183938 INFO  (SUITE-TriggerCooldownIntegrationTest-seed#[4336E26B99CC21FE]-worker) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49980/solr ready
   [junit4]   2> 1183953 INFO  (qtp2057559725-13238) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1183958 INFO  (qtp2057559725-13236) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1183960 INFO  (qtp798363708-13239) [n:127.0.0.1:49988_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1183960 INFO  (qtp798363708-13239) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1183961 DEBUG (qtp2057559725-13240) [n:127.0.0.1:49987_solr     ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1183965 INFO  (qtp2057559725-13240) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 1183976 DEBUG (zkCallback-9055-thread-2) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1183980 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1184007 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 1184009 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1184018 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
   [junit4]   2> 1184025 INFO  (qtp2057559725-13236) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1184026 INFO  (qtp798363708-13242) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1184026 DEBUG (qtp2057559725-13241) [n:127.0.0.1:49987_solr     ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1184032 INFO  (qtp2057559725-13241) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 1184032 DEBUG (zkCallback-9055-thread-2) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1184032 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1184036 INFO  (qtp2057559725-13238) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1184037 INFO  (qtp798363708-13244) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1184037 DEBUG (qtp2057559725-13240) [n:127.0.0.1:49987_solr     ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1184040 INFO  (qtp2057559725-13240) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 1184041 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1184041 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1184041 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1184042 INFO  (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.TriggerCooldownIntegrationTest TestTriggerAction init
   [junit4]   2> 1184042 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:49987_solr, 127.0.0.1:49988_solr]
   [junit4]   2> 1184042 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest$TestTriggerAction}], enabled=true}
   [junit4]   2> 1184043 DEBUG (zkCallback-9055-thread-2) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1184043 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1184044 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1184044 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1184044 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b4aca4{/solr,null,AVAILABLE}
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@1d1d6dde{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:49998}
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server Started @1184123ms
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49998}
   [junit4]   2> 1184045 ERROR (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1184045 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T06:54:42.972918Z
   [junit4]   2> 1184045 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 1184046 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1184048 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1184048 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1184049 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 3
   [junit4]   2> 1184049 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1184058 INFO  (zkConnectionManagerCallback-9081-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1184058 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1184059 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 1184059 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1184060 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1184171 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1184172 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5b01f580[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1184172 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5b01f580[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1184176 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6759f442[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1184177 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6759f442[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1184179 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49980/solr
   [junit4]   2> 1184183 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1184185 INFO  (zkConnectionManagerCallback-9092-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1184189 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1184212 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1184249 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:49998_solr as DOWN
   [junit4]   2> 1184263 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1184263 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49998_solr
   [junit4]   2> 1184269 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1184270 INFO  (zkCallback-9055-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1184270 INFO  (zkCallback-9078-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1184270 INFO  (zkCallback-9091-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1184275 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1184275 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1184308 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1184335 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49998.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1184344 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49998.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1184344 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49998.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1184348 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:49998_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001/node3
   [junit4]   2> 1184355 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=3
   [junit4]   2> 1184831 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 1
   [junit4]   2> 1185060 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1185060 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1185060 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:49998_solr at time 1194234729658223
   [junit4]   2> 1185833 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 3
   [junit4]   2> 1186067 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1186067 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1186067 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:49998_solr] added at times [1194234729658223], now=1194235736895674
   [junit4]   2> 1186067 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"43e2670d6f76fT9fuyw5dzph3td585na4smsr4l",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194234729658223,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194234729658223],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:49998_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1186067 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1186069 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e2670d6f76fT9fuyw5dzph3td585na4smsr4l",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194234729658223,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194234729658223],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194235737557349,
   [junit4]   2>     "nodeNames":["127.0.0.1:49998_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1186069 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:49998_solr
   [junit4]   2> 1186069 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"43e2670d6f76fT9fuyw5dzph3td585na4smsr4l",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194234729658223,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194234729658223],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194235737557349,
   [junit4]   2>     "nodeNames":["127.0.0.1:49998_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1186072 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e2670d6f76fT9fuyw5dzph3td585na4smsr4l",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194234729658223,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194234729658223],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194235737557349,
   [junit4]   2>     "nodeNames":["127.0.0.1:49998_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1186073 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 5000ms
   [junit4]   2> 1186073 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
   [junit4]   2> 1186073 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing took 3 ms for event id=43e2670d6f76fT9fuyw5dzph3td585na4smsr4l
   [junit4]   2> 1187081 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1187081 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1187081 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1187083 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1187083 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1187083 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1187083 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f57ae14{/solr,null,AVAILABLE}
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@7853ad48{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:50006}
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server Started @1187162ms
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50006}
   [junit4]   2> 1187084 ERROR (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1187084 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T06:54:46.011936Z
   [junit4]   2> 1187086 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1187089 INFO  (zkConnectionManagerCallback-9102-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1187089 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1187090 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1187200 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1187201 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7212b530[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1187201 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7212b530[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1187203 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@29a5d626[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1187203 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@29a5d626[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1187204 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49980/solr
   [junit4]   2> 1187205 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1187207 INFO  (zkConnectionManagerCallback-9113-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1187208 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1187218 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1187225 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:50006_solr as DOWN
   [junit4]   2> 1187227 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1187227 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50006_solr
   [junit4]   2> 1187230 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1187230 INFO  (zkCallback-9091-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1187230 INFO  (zkCallback-9078-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1187230 INFO  (zkCallback-9055-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1187230 INFO  (zkCallback-9112-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1187233 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1187233 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1187249 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1187268 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50006.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1187276 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50006.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1187276 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50006.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1187278 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50006_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001/node4
   [junit4]   2> 1191081 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1191081 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1191081 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 3
   [junit4]   2> 1191081 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:50006_solr at time 1194240751203050
   [junit4]   2> 1192240 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 4
   [junit4]   2> 1192241 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1192241 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1192241 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:50006_solr] added at times [1194240751203050], now=1194241910728606
   [junit4]   2> 1192241 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"43e27d7c072eaT9fuyw5dzph3td585na4smsr4m",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194240751203050,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194240751203050],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:50006_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1192241 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1192243 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e27d7c072eaT9fuyw5dzph3td585na4smsr4m",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194240751203050,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194240751203050],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194241910955655,
   [junit4]   2>     "nodeNames":["127.0.0.1:50006_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1192243 DEBUG (ScheduledTrigger-9061-thread-4) [     ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:50006_solr
   [junit4]   2> 1192243 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"43e27d7c072eaT9fuyw5dzph3td585na4smsr4m",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194240751203050,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194240751203050],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194241910955655,
   [junit4]   2>     "nodeNames":["127.0.0.1:50006_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1192255 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e27d7c072eaT9fuyw5dzph3td585na4smsr4m",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194240751203050,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194240751203050],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194241910955655,
   [junit4]   2>     "nodeNames":["127.0.0.1:50006_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1192255 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 5000ms
   [junit4]   2> 1192255 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
   [junit4]   2> 1192256 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing took 12 ms for event id=43e27d7c072eaT9fuyw5dzph3td585na4smsr4m
   [junit4]   2> 1194724 INFO  (qtp2057559725-13236) [n:127.0.0.1:49987_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1194727 INFO  (qtp798363708-13244) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=1
   [junit4]   2> 1194731 INFO  (qtp693569059-13295) [n:127.0.0.1:49998_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1194737 INFO  (qtp2027887326-13321) [n:127.0.0.1:50006_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=2
   [junit4]   2> 1194738 DEBUG (qtp798363708-13239) [n:127.0.0.1:49988_solr     ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1194741 INFO  (qtp798363708-13239) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=20
   [junit4]   2> 1194743 DEBUG (zkCallback-9055-thread-2) [     ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1194743 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1194744 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerCooldownPeriodSeconds from: 5 to: 7
   [junit4]   2> 1194745 INFO  (qtp798363708-13237) [n:127.0.0.1:49988_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1194746 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1194746 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1194746 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1194749 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1194750 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1194750 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread -- deactivating old nodeLost / nodeAdded markers
   [junit4]   2> 1194750 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1194752 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@549f9530{/solr,null,AVAILABLE}
   [junit4]   2> 1194754 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@287b2d3b{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:50056}
   [junit4]   2> 1194754 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.s.Server Started @1194832ms
   [junit4]   2> 1194754 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50056}
   [junit4]   2> 1194754 ERROR (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1194754 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1194754 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 1194755 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1194755 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1194755 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T06:54:53.682017Z
   [junit4]   2> 1194758 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1194761 DEBUG (OverseerAutoScalingTriggerThread-72135859289063430-127.0.0.1:49988_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1194763 INFO  (zkConnectionManagerCallback-9123-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1194763 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1194765 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1194888 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1194889 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2a5e635[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1194889 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2a5e635[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1194893 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4d64b787[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1194893 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4d64b787[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1194894 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49980/solr
   [junit4]   2> 1194896 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1194899 INFO  (zkConnectionManagerCallback-9134-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1194899 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1194914 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1194924 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:50056_solr as DOWN
   [junit4]   2> 1194927 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1194927 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50056_solr
   [junit4]   2> 1194933 INFO  (zkCallback-9112-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194933 INFO  (zkCallback-9091-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194933 INFO  (zkCallback-9055-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194933 INFO  (zkCallback-9133-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194935 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194935 INFO  (zkCallback-9078-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1194939 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1194939 WARN  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1194976 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1195005 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50056.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1195019 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50056.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1195019 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50056.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@229243a7
   [junit4]   2> 1195020 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [n:127.0.0.1:50056_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.autoscaling.TriggerCooldownIntegrationTest_4336E26B99CC21FE-001/tempDir-001/node5
   [junit4]   2> 1197266 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1197266 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5 and last live nodes: 4
   [junit4]   2> 1197266 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
   [junit4]   2> 1197266 DEBUG (ScheduledTrigger-9061-thread-2) [     ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:50056_solr at time 1194246935760292
   [junit4]   2> 1215041 INFO  (TEST-TriggerCooldownIntegrationTest.testCooldown-seed#[4336E26B99CC21FE]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testCooldown
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TriggerCooldownIntegrationTest -Dtests.method=testCooldown -Dtests.seed=4336E26B99CC21FE -Dtests.slow=true -Dtests.locale=es-DO -Dtests.timezone=NST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 31.1s J4 | TriggerCooldownIntegrationTest.testCooldown <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4336E26B99CC21FE:72888F8FE766540C]:0)
   [junit4]    > 	at org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest.testCooldown(TriggerCooldownIntegrationTest.java:155)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:832)
   [junit4]   2> 1215151 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=32451120
   [junit4]   2> 1215151 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:49988_solr
   [junit4]   2> 1215151 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1542114678
   [junit4]   2> 1215151 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:49987_solr
   [junit4]   2> 1215152 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1548492365
   [junit4]   2> 1215152 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:49998_solr
   [junit4]   2> 1215152 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=899204407
   [junit4]   2> 1215152 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:50006_solr
   [junit4]   2> 1215155 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 1215155 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:50006_solr as DOWN
   [junit4]   2> 1215157 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 1215157 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:49987_solr as DOWN
   [junit4]   2> 1215157 INFO  (zkCallback-9112-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1215158 INFO  (zkCallback-9133-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1215158 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (3)
   [junit4]   2> 1215159 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 1215159 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:49988_solr as DOWN
   [junit4]   2> 1215159 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 1215159 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:49998_solr as DOWN
   [junit4]   2> 1215159 INFO  (zkCallback-9091-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (3)
   [junit4]   2> 1215160 INFO  (zkCallback-9055-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (3)
   [junit4]   2> 1215160 INFO  (zkCallback-9133-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1215160 INFO  (zkCallback-9112-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1215165 INFO  (jetty-closer-9143-thread-5) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1535399152
   [junit4]   2> 1215165 INFO  (jetty-closer-9143-thread-5) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:50056_solr
   [junit4]   2> 1215170 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3 and last live nodes: 5
   [junit4]   2> 1215170 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:50006_solr
   [junit4]   2> 1215170 DEBUG (ScheduledTrigger-9061-thread-1) [     ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:49998_solr
   [junit4]   2> 1215175 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1215175 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1215175 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:50056_solr] added at times [1194246935760292], now=1194264844945309
   [junit4]   2> 1215175 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"43e2948614da4T9fuyw5dzph3td585na4smsr4n",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194246935760292,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194246935760292],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "nodeNames":["127.0.0.1:50056_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1215175 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1215179 INFO  (jetty-closer-9143-thread-5) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 1215179 INFO  (jetty-closer-9143-thread-5) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:50056_solr as DOWN
   [junit4]   2> 1215181 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 1215181 INFO  (jetty-closer-9143-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e45154e: rootName = solr_50006, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@68557c83
   [junit4]   2> 1215182 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 1215182 INFO  (jetty-closer-9143-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2563853: rootName = solr_49988, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@6e0c4b4b
   [junit4]   2> 1215183 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 1215183 INFO  (jetty-closer-9143-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@365573bd: rootName = solr_49987, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@22fefa7e
   [junit4]   2> 1215184 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 1215184 INFO  (jetty-closer-9143-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20c053fb: rootName = solr_49998, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@3333aa08
   [junit4]   2> 1215185 INFO  (zkCallback-9091-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 1215186 INFO  (zkCallback-9055-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 1215186 INFO  (zkCallback-9112-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 1215186 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e2948614da4T9fuyw5dzph3td585na4smsr4n",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194246935760292,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194246935760292],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194264845103092,
   [junit4]   2>     "nodeNames":["127.0.0.1:50056_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1215186 INFO  (zkCallback-9133-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 1215187 DEBUG (ScheduledTrigger-9061-thread-3) [     ] o.a.s.c.a.NodeAddedTrigger Removing new node from tracking: 127.0.0.1:50056_solr
   [junit4]   2> 1215187 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"43e2948614da4T9fuyw5dzph3td585na4smsr4n",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194246935760292,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194246935760292],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194264845103092,
   [junit4]   2>     "nodeNames":["127.0.0.1:50056_solr"],
   [junit4]   2>     "replicaType":"NRT"}}
   [junit4]   2> 1215187 INFO  (zkCallback-9042-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 1215193 DEBUG (AutoscalingActionExecutor-9062-thread-1) [     ] o.a.s.c.a.SystemLogListener Collection .system missing, skip sending event {
   [junit4]   2>   "id":"43e2948614da4T9fuyw5dzph3td585na4smsr4n",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":1194246935760292,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194246935760292],
   [junit4]   2>  

[...truncated too long message...]

ed by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
   [junit4]   2>   "id":"43e2fdb268a04T9fuyw5dzph3td585na4smsrzc",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":1194275167963652,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1194275167963652],
   [junit4]   2>     "preferredOperation":"movereplica",
   [junit4]   2>     "_enqueue_time_":1194276190652995,
   [junit4]   2>     "nodeNames":["127.0.0.1:50188_solr"]}}
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:186) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:328) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: ExecutePlanAction was interrupted
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:179) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:328) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> Caused by: java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method) ~[?:?]
   [junit4]   2> 	at org.apache.solr.common.util.TimeSource$NanoTimeSource.sleep(TimeSource.java:104) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.waitForTaskToFinish(ExecutePlanAction.java:227) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:328) ~[java/:?]
   [junit4]   2> 	... 6 more
   [junit4]   2> 1228345 DEBUG (AutoscalingActionExecutor-6836-thread-1) [     ] o.a.s.c.a.ScheduledTriggers -- processing took 1834 ms for event id=43e2fdb268a04T9fuyw5dzph3td585na4smsrzc
   [junit4]   2> 1228345 DEBUG (closeThreadPool-6851-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1228359 DEBUG (closeThreadPool-6851-thread-1) [     ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1228359 DEBUG (closeThreadPool-6851-thread-1) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1228359 INFO  (closeThreadPool-6851-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72135861731852293-127.0.0.1:50187_solr-n_0000000001) closing
   [junit4]   2> 1228437 INFO  (jetty-closer-6845-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72135861731852293-127.0.0.1:50187_solr-n_0000000001) closing
   [junit4]   2> 1228443 INFO  (jetty-closer-6845-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@157a63aa{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
   [junit4]   2> 1228443 INFO  (jetty-closer-6845-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@24a5461a{/solr,null,UNAVAILABLE}
   [junit4]   2> 1228443 INFO  (jetty-closer-6845-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1228450 INFO  (TEST-ExecutePlanActionTest.testIntegration-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 1228683 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections/testIntegration/terms/shard1
   [junit4]   2> 	3	/solr/aliases.json
   [junit4]   2> 	2	/solr/packages.json
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 	2	/solr/configs/conf
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	34	/solr/collections/testIntegration/state.json
   [junit4]   2> 	4	/solr/autoscaling.json
   [junit4]   2> 	3	/solr/clusterprops.json
   [junit4]   2> 	2	/solr/collections/testIntegration/leader_elect/shard1/election/72135861731852293-core_node4-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	32	/solr/overseer/queue
   [junit4]   2> 	8	/solr/live_nodes
   [junit4]   2> 	7	/solr/overseer/collection-queue-work
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	2	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	2	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	2	/solr/autoscaling/events/node_lost_trigger
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> 1228690 INFO  (TEST-ExecutePlanActionTest.testIntegration-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:50182
   [junit4]   2> 1228690 INFO  (TEST-ExecutePlanActionTest.testIntegration-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:50182
   [junit4]   2> 1228690 INFO  (TEST-ExecutePlanActionTest.testIntegration-seed#[4336E26B99CC21FE]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 50182
   [junit4]   2> 1228691 INFO  (TEST-ExecutePlanActionTest.testIntegration-seed#[4336E26B99CC21FE]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testIntegration
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.autoscaling.ExecutePlanActionTest_4336E26B99CC21FE-001
   [junit4]   2> Jun 20, 2020 6:55:27 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {}, docValues:{}, maxPointsInLeafNode=1556, maxMBSortInHeap=5.9645397816953825, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@477e43bf), locale=agq, timezone=America/St_Thomas
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 14.0.1 (64-bit)/cpus=6,threads=1,free=109136896,total=365953024
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudPseudoReturnFields, TestObjectReleaseTracker, TestCloudConsistency, ScheduledTriggerIntegrationTest, OverseerSolrResponseUnsafeSerializationTest, MetricsHandlerTest, TestRestoreCore, DateRangeFieldTest, HdfsBasicDistributedZkTest, CachingDirectoryFactoryTest, MigrateRouteKeyTest, TestSimExtremeIndexing, SplitShardTest, TestBinaryResponseWriter, TestWaitForStateWithJettyShutdowns, MaxSizeAutoCommitTest, TestDistributedTracing, ZkStateReaderTest, TestFaceting, TestNestedUpdateProcessor, TestFieldCollectionResource, TestCollectionAPI, TestHdfsBackupRestoreCore, HdfsDirectoryFactoryTest, SuggesterFSTTest, TestGroupingSearch, CdcrRequestHandlerTest, TestSegmentSorting, TestDelegationWithHadoopAuth, IndexSizeTriggerTest, TestDocSet, TestSimpleTrackingShardHandler, PeerSyncReplicationTest, SolrIndexMetricsTest, CollectionTooManyReplicasTest, TestSubQueryTransformer, ChaosMonkeyNothingIsSafeTest, TestLMDirichletSimilarityFactory, ExternalFileFieldSortTest, DebugComponentTest, TestPartialUpdateDeduplication, SpellCheckComponentTest, HttpPartitionWithTlogReplicasTest, TestSimPolicyCloud, TestCloudJSONFacetSKG, TestUseDocValuesAsStored, VersionInfoTest, VMParamsZkACLAndCredentialsProvidersTest, OutputWriterTest, TestTlogReplayVsRecovery, TestMergePolicyConfig, TestPerFieldSimilarityWithDefaultOverride, FastVectorHighlighterTest, TestBM25SimilarityFactory, TestReload, QueryElevationComponentTest, TestExactSharedStatsCache, TestPrepRecovery, HdfsUnloadDistributedZkTest, TestFiltering, TestConfigSetProperties, QueryEqualityTest, DistributedFacetPivotLongTailTest, SolrIndexSearcherTest, CollectionPropsTest, CleanupOldIndexTest, HdfsRecoveryZkTest, RandomizedTaggerTest, ZookeeperStatusHandlerTest, TestPivotHelperCode, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSimClusterStateProvider, XsltUpdateRequestHandlerTest, TestRandomFlRTGCloud, CurrencyRangeFacetCloudTest, AnalysisErrorHandlingTest, ExecutePlanActionTest]
   [junit4] Completed [381/910 (2!)] on J3 in 63.12s, 4 tests, 1 failure <<< FAILURES!

[...truncated 46603 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6188/consoleText

[repro] Revision: 591d844e806c4887974d887097fe8401f66f7049

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC"
[repro] Repro line:  ant test  -Dtestcase=TriggerCooldownIntegrationTest -Dtests.method=testCooldown -Dtests.seed=4336E26B99CC21FE -Dtests.slow=true -Dtests.locale=es-DO -Dtests.timezone=NST -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=ExecutePlanActionTest -Dtests.method=testTaskTimeout -Dtests.seed=4336E26B99CC21FE -Dtests.slow=true -Dtests.locale=agq -Dtests.timezone=America/St_Thomas -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       ExecutePlanActionTest
[repro]       TriggerCooldownIntegrationTest
[repro] ant compile-test

[...truncated 2471 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.ExecutePlanActionTest|*.TriggerCooldownIntegrationTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=4336E26B99CC21FE -Dtests.slow=true -Dtests.locale=agq -Dtests.timezone=America/St_Thomas -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 116 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.ExecutePlanActionTest
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.TriggerCooldownIntegrationTest
[repro] Exiting with code 0

[...truncated 73 lines...]

[JENKINS-Experimental-GC] Lucene-Solr-master-MacOSX (64bit/jdk-14.0.1) - Build # 6187 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6187/
Java: 64bit/jdk-14.0.1 -XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ZkShardTermsTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ZkShardTermsTest:     1) Thread[id=897, name=SessionTracker, state=TIMED_WAITING, group=TGRP-ZkShardTermsTest]         at java.base@14.0.1/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ZkShardTermsTest: 
   1) Thread[id=897, name=SessionTracker, state=TIMED_WAITING, group=TGRP-ZkShardTermsTest]
        at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([8DFCE42228B528C4]:0)


FAILED:  org.apache.solr.cloud.autoscaling.RestoreTriggerStateTest.testEventFromRestoredState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
	at __randomizedtesting.SeedInfo.seed([8DFCE42228B528C4:8DCA50E201B88CAE]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.autoscaling.RestoreTriggerStateTest.testEventFromRestoredState(RestoreTriggerStateTest.java:113)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)




Build Log:
[...truncated 12223 lines...]
   [junit4] Suite: org.apache.solr.cloud.ZkShardTermsTest
   [junit4]   2> 135972 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 135972 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 135974 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/data-dir-7-001
   [junit4]   2> 135975 WARN  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 135975 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 135988 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 135989 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001
   [junit4]   2> 135989 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 136015 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 136016 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 136258 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer start zk server on port: 49366
   [junit4]   2> 136258 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:49366
   [junit4]   2> 136258 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:49366
   [junit4]   2> 136258 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 49366
   [junit4]   2> 136306 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 136403 INFO  (zkConnectionManagerCallback-654-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 136403 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 136415 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 136464 INFO  (zkConnectionManagerCallback-656-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 136465 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 136467 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 136475 INFO  (zkConnectionManagerCallback-658-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 136475 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 136613 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 136613 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 136614 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 136620 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 136620 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 136621 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 136622 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57bf2843{/solr,null,AVAILABLE}
   [junit4]   2> 136623 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@79189ad8{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:49373}
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.e.j.s.Server Started @136796ms
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49373}
   [junit4]   2> 136624 ERROR (jetty-launcher-659-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 136624 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-20T01:47:07.538801Z
   [junit4]   2> 136626 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 136636 INFO  (zkConnectionManagerCallback-661-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 136636 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 136639 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 136766 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 136769 WARN  (jetty-launcher-659-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@15848042[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 136769 WARN  (jetty-launcher-659-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@15848042[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 136778 WARN  (jetty-launcher-659-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1228e18f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 136778 WARN  (jetty-launcher-659-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1228e18f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 136779 INFO  (jetty-launcher-659-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49366/solr
   [junit4]   2> 136782 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 136794 INFO  (zkConnectionManagerCallback-672-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 136795 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137279 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49373_solr
   [junit4]   2> 137281 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.Overseer Overseer (id=72134649812418564-127.0.0.1:49373_solr-n_0000000000) starting
   [junit4]   2> 137307 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:49373_solr
   [junit4]   2> 137310 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49373_solr
   [junit4]   2> 137317 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 137341 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 137342 WARN  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 137371 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 137411 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 137425 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 137425 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 137427 INFO  (jetty-launcher-659-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1
   [junit4]   2> 137451 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=1
   [junit4]   2> 137452 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 137462 INFO  (zkConnectionManagerCallback-689-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 137463 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 137478 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 137494 INFO  (SUITE-ZkShardTermsTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49366/solr ready
   [junit4]   2> 137592 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testCoreTermWatcher
   [junit4]   2> 137644 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/coreTermWatcher/terms/shard1 to Terms{values={leader=0}, version=0}
   [junit4]   2> 137686 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/coreTermWatcher/terms/shard1 to Terms{values={replica=0, leader=0}, version=1}
   [junit4]   2> 137696 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 137704 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/coreTermWatcher/terms/shard1 to Terms{values={replica=0, leader=1}, version=2}
   [junit4]   2> 137809 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/coreTermWatcher/terms/shard1 to Terms{values={replica=1, leader=1}, version=3}
   [junit4]   2> 137811 INFO  (TEST-ZkShardTermsTest.testCoreTermWatcher-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testCoreTermWatcher
   [junit4]   2> 138099 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testRecoveringFlag
   [junit4]   2> 138127 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=0}, version=0}
   [junit4]   2> 138129 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=0, replica2=0}, version=1}
   [junit4]   2> 138133 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=1, replica2=0}, version=2}
   [junit4]   2> 138141 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=1, replica2_recovering=0, replica2=1}, version=3}
   [junit4]   2> 138149 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=1, replica2=1}, version=4}
   [junit4]   2> 138168 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=2, replica2=1}, version=5}
   [junit4]   2> 138176 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=2, replica2_recovering=1, replica2=2}, version=6}
   [junit4]   2> 138182 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=3, replica2_recovering=1, replica2=2}, version=7}
   [junit4]   2> 138192 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=3, replica2=2}, version=8}
   [junit4]   2> 138207 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=3, replica2_recovering=2, replica2=3}, version=9}
   [junit4]   2> 138217 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=3, replica2=3}, version=10}
   [junit4]   2> 138225 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=4, replica2=3}, version=11}
   [junit4]   2> 138244 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=4, replica2_recovering=3, replica2=4}, version=12}
   [junit4]   2> 138247 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=5, replica2_recovering=3, replica2=4}, version=13}
   [junit4]   2> 138249 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=5, replica2_recovering=3, replica2=5}, version=14}
   [junit4]   2> 138253 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlag/terms/shard1 to Terms{values={replica1=5, replica2=5}, version=15}
   [junit4]   2> 138254 INFO  (TEST-ZkShardTermsTest.testRecoveringFlag-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testRecoveringFlag
   [junit4]   2> 138521 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testSetTermEqualsToLeader
   [junit4]   2> 138580 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermEqualsToLeader/terms/shard1 to Terms{values={leader=0}, version=0}
   [junit4]   2> 138595 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 138600 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermEqualsToLeader/terms/shard1 to Terms{values={replica=0, leader=0}, version=1}
   [junit4]   2> 138606 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 138624 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermEqualsToLeader/terms/shard1 to Terms{values={replica=0, leader=1}, version=2}
   [junit4]   2> 138632 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermEqualsToLeader/terms/shard1 to Terms{values={replica=1, leader=1}, version=3}
   [junit4]   2> 138635 INFO  (TEST-ZkShardTermsTest.testSetTermEqualsToLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testSetTermEqualsToLeader
   [junit4]   2> 138840 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testReplicaCanBecomeLeader
   [junit4]   2> 138945 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaCanBecomeLeader/terms/shard1 to Terms{values={leader=0}, version=0}
   [junit4]   2> 138951 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 138978 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaCanBecomeLeader/terms/shard1 to Terms{values={replica=0, leader=0}, version=1}
   [junit4]   2> 138985 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 139003 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaCanBecomeLeader/terms/shard1 to Terms{values={replica=0, leader=1}, version=2}
   [junit4]   2> 139141 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaCanBecomeLeader/terms/shard1 to Terms{values={replica=1, leader=1, replica_recovering=0}, version=3}
   [junit4]   2> 139245 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaCanBecomeLeader/terms/shard1 to Terms{values={leader=1, replica=1}, version=4}
   [junit4]   2> 139246 INFO  (TEST-ZkShardTermsTest.testReplicaCanBecomeLeader-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testReplicaCanBecomeLeader
   [junit4]   2> 139285 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testCoreRemovalWhileRecovering
   [junit4]   2> 139294 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlagRemoval/terms/shard1 to Terms{values={replica1_rem=0}, version=0}
   [junit4]   2> 139295 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlagRemoval/terms/shard1 to Terms{values={replica1_rem=0, replica2_rem=0}, version=1}
   [junit4]   2> 139296 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlagRemoval/terms/shard1 to Terms{values={replica1_rem=1, replica2_rem=0}, version=2}
   [junit4]   2> 139299 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlagRemoval/terms/shard1 to Terms{values={replica1_rem=1, replica2_rem_recovering=0, replica2_rem=1}, version=3}
   [junit4]   2> 139301 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/recoveringFlagRemoval/terms/shard1 to Terms{values={replica1_rem=1}, version=4}
   [junit4]   2> 139302 INFO  (TEST-ZkShardTermsTest.testCoreRemovalWhileRecovering-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testCoreRemovalWhileRecovering
   [junit4]   2> 139387 INFO  (TEST-ZkShardTermsTest.testParticipationOfReplicas-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testParticipationOfReplicas
   [junit4]   2> 139403 INFO  (TEST-ZkShardTermsTest.testParticipationOfReplicas-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={replica1=0}, version=0}
   [junit4]   2> 139407 INFO  (TEST-ZkShardTermsTest.testParticipationOfReplicas-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={replica1=0, replica2=0}, version=1}
   [junit4]   2> 139410 INFO  (TEST-ZkShardTermsTest.testParticipationOfReplicas-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={replica1=1, replica2=0}, version=2}
   [junit4]   2> 139418 INFO  (qtp2011130451-927) [n:127.0.0.1:49373_solr     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&maxShardsPerNode=1000&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&createNodeSet=127.0.0.1:49373_solr&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 139428 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 139433 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/collection1.AUTOCREATED/managed-schema
   [junit4]   2> 139439 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/collection1.AUTOCREATED/protwords.txt
   [junit4]   2> 139447 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/collection1.AUTOCREATED/lang/contractions_it.txt
   [junit4]   2> 139453 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/collection1.AUTOCREATED/lang/contractions_ca.txt
   [junit4]   2> 139459 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/collection1.AUTOCREATED/lang/stemdict_nl.txt
   [junit4]   2> 139465 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/collection1.AUTOCREATED/lang/stopwords_no.txt
   [junit4]   2> 139471 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/collection1.AUTOCREATED/lang/stopwords_hy.txt
   [junit4]   2> 139478 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/collection1.AUTOCREATED/lang/stopwords_id.txt
   [junit4]   2> 139483 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_et.txt to /configs/collection1.AUTOCREATED/lang/stopwords_et.txt
   [junit4]   2> 139489 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/collection1.AUTOCREATED/lang/stopwords_da.txt
   [junit4]   2> 139496 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/collection1.AUTOCREATED/lang/stopwords_hi.txt
   [junit4]   2> 139502 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ga.txt
   [junit4]   2> 139508 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/collection1.AUTOCREATED/lang/stopwords_pt.txt
   [junit4]   2> 139514 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ja.txt
   [junit4]   2> 139521 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/collection1.AUTOCREATED/lang/stopwords_hu.txt
   [junit4]   2> 139576 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/collection1.AUTOCREATED/lang/stopwords_el.txt
   [junit4]   2> 139641 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ru.txt
   [junit4]   2> 139709 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/collection1.AUTOCREATED/lang/stopwords_tr.txt
   [junit4]   2> 139784 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ar.txt
   [junit4]   2> 139850 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/collection1.AUTOCREATED/lang/userdict_ja.txt
   [junit4]   2> 139894 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/collection1.AUTOCREATED/lang/stopwords_eu.txt
   [junit4]   2> 139917 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/collection1.AUTOCREATED/lang/stopwords_it.txt
   [junit4]   2> 139937 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/collection1.AUTOCREATED/lang/stopwords_cz.txt
   [junit4]   2> 139971 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/collection1.AUTOCREATED/lang/stoptags_ja.txt
   [junit4]   2> 139987 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/collection1.AUTOCREATED/lang/contractions_fr.txt
   [junit4]   2> 140005 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/collection1.AUTOCREATED/lang/stopwords_de.txt
   [junit4]   2> 140018 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/collection1.AUTOCREATED/lang/stopwords_fa.txt
   [junit4]   2> 140041 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/collection1.AUTOCREATED/lang/contractions_ga.txt
   [junit4]   2> 140074 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ca.txt
   [junit4]   2> 140105 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/collection1.AUTOCREATED/lang/stopwords_nl.txt
   [junit4]   2> 140117 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/collection1.AUTOCREATED/lang/stopwords_bg.txt
   [junit4]   2> 140131 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/collection1.AUTOCREATED/lang/stopwords_es.txt
   [junit4]   2> 140151 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/collection1.AUTOCREATED/lang/stopwords_lv.txt
   [junit4]   2> 140204 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/collection1.AUTOCREATED/lang/stopwords_en.txt
   [junit4]   2> 140250 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/collection1.AUTOCREATED/lang/stopwords_ro.txt
   [junit4]   2> 140280 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/collection1.AUTOCREATED/lang/stopwords_th.txt
   [junit4]   2> 140319 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/collection1.AUTOCREATED/lang/stopwords_fr.txt
   [junit4]   2> 140343 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/collection1.AUTOCREATED/lang/stopwords_sv.txt
   [junit4]   2> 140383 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/collection1.AUTOCREATED/lang/stopwords_fi.txt
   [junit4]   2> 140401 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/collection1.AUTOCREATED/lang/hyphenations_ga.txt
   [junit4]   2> 140411 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/collection1.AUTOCREATED/lang/stopwords_gl.txt
   [junit4]   2> 140430 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/collection1.AUTOCREATED/solrconfig.xml
   [junit4]   2> 140448 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/collection1.AUTOCREATED/synonyms.txt
   [junit4]   2> 140466 INFO  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/collection1.AUTOCREATED/stopwords.txt
   [junit4]   2> 140641 WARN  (OverseerThreadFactory-679-thread-1-processing-n:127.0.0.1:49373_solr) [n:127.0.0.1:49373_solr     ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 2 on collection collection1 is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 140670 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 140671 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=1
   [junit4]   2> 140750 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:49373/solr",
   [junit4]   2>   "node_name":"127.0.0.1:49373_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 140808 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:49373/solr",
   [junit4]   2>   "node_name":"127.0.0.1:49373_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 140831 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:49373/solr",
   [junit4]   2>   "node_name":"127.0.0.1:49373_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 140903 INFO  (OverseerStateUpdate-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n7",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:49373/solr",
   [junit4]   2>   "node_name":"127.0.0.1:49373_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 141120 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr    x:collection1_shard2_replica_n4 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 141120 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr    x:collection1_shard1_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 141120 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr    x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 141124 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr    x:collection1_shard2_replica_n7 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard2_replica_n7&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 141234 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 141248 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 141248 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 141260 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 141346 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.s.IndexSchema Schema name=default-config
   [junit4]   2> 141357 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.s.IndexSchema Schema name=default-config
   [junit4]   2> 141359 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.IndexSchema Schema name=default-config
   [junit4]   2> 141367 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=default-config
   [junit4]   2> 143141 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 143141 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset collection1.AUTOCREATED, trusted=true
   [junit4]   2> 143152 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 143155 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 143155 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from configset collection1.AUTOCREATED, trusted=true
   [junit4]   2> 143155 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard1_replica_n1/data/]
   [junit4]   2> 143209 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 143211 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard1_replica_n2/data/]
   [junit4]   2> 143259 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 143259 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n4' using configuration from configset collection1.AUTOCREATED, trusted=true
   [junit4]   2> 143260 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.core.collection1.shard2.replica_n4' (registry 'solr.core.collection1.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 143260 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard2_replica_n4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard2_replica_n4/data/]
   [junit4]   2> 143369 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 143369 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n7' using configuration from configset collection1.AUTOCREATED, trusted=true
   [junit4]   2> 143370 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49373.solr.core.collection1.shard2.replica_n7' (registry 'solr.core.collection1.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35f953b8
   [junit4]   2> 143370 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n7] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard2_replica_n7], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.ZkShardTermsTest_8DFCE42228B528C4-001/tempDir-001/node1/collection1_shard2_replica_n7/data/]
   [junit4]   2> 144444 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 144444 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 144444 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 144444 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 144446 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 144446 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 144447 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 144447 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 144448 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 144448 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 144448 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 144450 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 144454 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 144454 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 144458 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/collection1.AUTOCREATED
   [junit4]   2> 144459 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 144459 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 144462 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/collection1.AUTOCREATED
   [junit4]   2> 144463 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/collection1.AUTOCREATED
   [junit4]   2> 144467 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/collection1.AUTOCREATED
   [junit4]   2> 144468 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/collection1.AUTOCREATED
   [junit4]   2> 144468 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/collection1.AUTOCREATED
   [junit4]   2> 144470 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/collection1.AUTOCREATED
   [junit4]   2> 144471 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/collection1.AUTOCREATED
   [junit4]   2> 144472 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/collection1.AUTOCREATED/managed-schema
   [junit4]   2> 144473 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/collection1.AUTOCREATED/managed-schema
   [junit4]   2> 144473 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/collection1.AUTOCREATED/managed-schema
   [junit4]   2> 144473 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/collection1.AUTOCREATED/managed-schema
   [junit4]   2> 144476 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 144477 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 144477 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 144478 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 144480 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 144480 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 144481 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 144486 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.DirectSolrSpellChecker init: {name=default, field=_text_, classname=solr.DirectSolrSpellChecker, distanceMeasure=internal, accuracy=0.5, maxEdits=2, minPrefix=1, maxInspections=5, minQueryLength=4, maxQueryFrequency=0.01}
   [junit4]   2> 144488 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.s.DirectSolrSpellChecker init: {name=default, field=_text_, classname=solr.DirectSolrSpellChecker, distanceMeasure=internal, accuracy=0.5, maxEdits=2, minPrefix=1, maxInspections=5, minQueryLength=4, maxQueryFrequency=0.01}
   [junit4]   2> 144488 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 144488 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.DirectSolrSpellChecker init: {name=default, field=_text_, classname=solr.DirectSolrSpellChecker, distanceMeasure=internal, accuracy=0.5, maxEdits=2, minPrefix=1, maxInspections=5, minQueryLength=4, maxQueryFrequency=0.01}
   [junit4]   2> 144495 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.s.DirectSolrSpellChecker init: {name=default, field=_text_, classname=solr.DirectSolrSpellChecker, distanceMeasure=internal, accuracy=0.5, maxEdits=2, minPrefix=1, maxInspections=5, minQueryLength=4, maxQueryFrequency=0.01}
   [junit4]   2> 144500 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 144500 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669980629671870464
   [junit4]   2> 144501 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 144501 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669980629672919040
   [junit4]   2> 144507 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 144509 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669980629681307648
   [junit4]   2> 144511 INFO  (searcherExecutor-695-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n4 c:collection1 s:shard2 r:core_node6) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 144511 INFO  (searcherExecutor-695-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n4 c:collection1 s:shard2 r:core_node6) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 144511 INFO  (searcherExecutor-691-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 144511 INFO  (searcherExecutor-691-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 144513 INFO  (searcherExecutor-695-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n4 c:collection1 s:shard2 r:core_node6) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrCore [collection1_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 144513 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 144514 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1669980629686550528
   [junit4]   2> 144515 INFO  (searcherExecutor-691-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 144519 INFO  (searcherExecutor-693-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n2 c:collection1 s:shard1 r:core_node5) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 144519 INFO  (searcherExecutor-693-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n2 c:collection1 s:shard1 r:core_node5) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 144521 INFO  (searcherExecutor-693-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard1_replica_n2 c:collection1 s:shard1 r:core_node5) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore [collection1_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 144522 INFO  (searcherExecutor-697-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n7 c:collection1 s:shard2 r:core_node8) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 144522 INFO  (searcherExecutor-697-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n7 c:collection1 s:shard2 r:core_node8) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 144527 INFO  (searcherExecutor-697-thread-1-processing-n:127.0.0.1:49373_solr x:collection1_shard2_replica_n7 c:collection1 s:shard2 r:core_node8) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.SolrCore [collection1_shard2_replica_n7]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 144539 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 144539 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 144571 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node8=0}, version=0}
   [junit4]   2> 144573 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node6=0, core_node8=0}, version=1}
   [junit4]   2> 144574 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
   [junit4]   2> 144575 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
   [junit4]   2> 144577 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
   [junit4]   2> 144579 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 144598 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 144599 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 144600 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49373/solr/collection1_shard1_replica_n1/
   [junit4]   2> 144608 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=http://127.0.0.1:49373/solr  START replicas=[http://127.0.0.1:49373/solr/collection1_shard1_replica_n2/] nUpdates=100
   [junit4]   2> 144610 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=http://127.0.0.1:49373/solr  DONE. We have no versions. sync failed.
   [junit4]   2> 144617 INFO  (qtp2011130451-979) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.S.Request [collection1_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 144619 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 144619 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 144619 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72134649812418564-core_node3-n_0000000000
   [junit4]   2> 144631 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 144631 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 144631 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49373/solr/collection1_shard2_replica_n7/
   [junit4]   2> 144633 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49373/solr/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 144636 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n7 url=http://127.0.0.1:49373/solr  START replicas=[http://127.0.0.1:49373/solr/collection1_shard2_replica_n4/] nUpdates=100
   [junit4]   2> 144640 INFO  (qtp2011130451-979) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node6 x:collection1_shard2_replica_n4 ] o.a.s.c.S.Request [collection1_shard2_replica_n4]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 144640 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n7 url=http://127.0.0.1:49373/solr  DONE. We have no versions. sync failed.
   [junit4]   2> 144642 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 144642 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 144645 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72134649812418564-core_node8-n_0000000000
   [junit4]   2> 144656 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49373/solr/collection1_shard2_replica_n7/ shard2
   [junit4]   2> 144775 INFO  (zkCallback-671-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 144782 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 144784 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 144796 INFO  (qtp2011130451-926) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3676
   [junit4]   2> 144799 INFO  (qtp2011130451-966) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard2_replica_n7&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3674
   [junit4]   2> 144901 INFO  (zkCallback-671-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 144901 INFO  (zkCallback-671-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 145621 INFO  (qtp2011130451-929) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4501
   [junit4]   2> 145640 INFO  (qtp2011130451-928) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=collection1.AUTOCREATED&newCollection=true&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4520
   [junit4]   2> 145651 INFO  (qtp2011130451-927) [n:127.0.0.1:49373_solr     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 145743 INFO  (zkCallback-671-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 145743 INFO  (zkCallback-671-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 145746 INFO  (qtp2011130451-927) [n:127.0.0.1:49373_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&maxShardsPerNode=1000&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&createNodeSet=127.0.0.1:49373_solr&wt=javabin&version=2} status=0 QTime=6328
   [junit4]   2> 145769 INFO  (TEST-ZkShardTermsTest.testParticipationOfReplicas-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testParticipationOfReplicas
   [junit4]   2> 145849 INFO  (TEST-ZkShardTermsTest.testEnsureTermsIsHigher-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testEnsureTermsIsHigher
   [junit4]   2> 145851 INFO  (TEST-ZkShardTermsTest.testEnsureTermsIsHigher-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testEnsureTermsIsHigher
   [junit4]   2> 145973 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testSetTermToZero
   [junit4]   2> 145988 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermToZero/terms/shard1 to Terms{values={leader=0}, version=0}
   [junit4]   2> 145993 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermToZero/terms/shard1 to Terms{values={replica=0, leader=0}, version=1}
   [junit4]   2> 145997 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermToZero/terms/shard1 to Terms{values={replica=0, leader=1}, version=2}
   [junit4]   2> 146000 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/setTermToZero/terms/shard1 to Terms{values={replica=0, leader=0}, version=3}
   [junit4]   2> 146002 INFO  (TEST-ZkShardTermsTest.testSetTermToZero-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testSetTermToZero
   [junit4]   2> 146191 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testRegisterTerm
   [junit4]   2> 146220 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/registerTerm/terms/shard1 to Terms{values={rep1=0}, version=0}
   [junit4]   2> 146223 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 146231 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/registerTerm/terms/shard1 to Terms{values={rep2=0, rep1=0}, version=1}
   [junit4]   2> 146256 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/registerTerm/terms/shard1 to Terms{values={rep2=0, rep1=1}, version=2}
   [junit4]   2> 146362 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/registerTerm/terms/shard1 to Terms{values={rep2=1, rep1=1}, version=3}
   [junit4]   2> 146368 INFO  (TEST-ZkShardTermsTest.testRegisterTerm-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testRegisterTerm
   [junit4]   2> 146553 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testRaceConditionOnUpdates
   [junit4]   2> 146571 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceConditionOnUpdates/terms/shard1 to Terms{values={rep1=0}, version=0}
   [junit4]   2> 146586 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceConditionOnUpdates/terms/shard1 to Terms{values={rep2=0, rep1=0}, version=1}
   [junit4]   2> 146606 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceConditionOnUpdates/terms/shard1 to Terms{values={rep2=0, rep1=0, rep3=0}, version=2}
   [junit4]   2> 146624 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceConditionOnUpdates/terms/shard1 to Terms{values={rep2=0, rep1=0, rep4=0, rep3=0}, version=3}
   [junit4]   2> 146653 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceConditionOnUpdates/terms/shard1 to Terms{values={rep2=0, rep1=0, leader=0, rep4=0, rep3=0}, version=4}
   [junit4]   2> 147562 INFO  (OverseerCollectionConfigSetProcessor-72134649812418564-127.0.0.1:49373_solr-n_0000000000) [n:127.0.0.1:49373_solr     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 156807 INFO  (TEST-ZkShardTermsTest.testRaceConditionOnUpdates-seed#[8DFCE42228B528C4]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testRaceConditionOnUpdates
   [junit4]   2> 156923 INFO  (jetty-closer-714-thread-1) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=97148831
   [junit4]   2> 156923 INFO  (jetty-closer-714-thread-1) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:49373_solr
   [junit4]   2> 156924 INFO  (jetty-closer-714-thread-1) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 156924 INFO  (jetty-closer-714-thread-1) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:49373_solr as DOWN
   [junit4]   2> 156931 INFO  (coreCloseExecutor-718-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.c.SolrCore [collection1_shard2_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c517959
   [junit4]   2> 156931 INFO  (coreCloseExecutor-718-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n4 tag=SolrCore@1c517959
   [junit4]   2> 156931 INFO  (coreCloseExecutor-718-thread-1) [n:127.0.0.1:49373_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@21386199: rootName = solr_49373, domain = solr.core.collection1.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n4/com.codahale.metrics.MetricRegistry@3eaf9e95
   [junit4]   2> 156932 INFO  (coreCloseExecutor-718-thread-2) [n:127.0.0.1:49373_solr     ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1454bcf5
   [junit4]   2> 156933 INFO  (coreCloseExecutor-718-thread-3) [n:127.0.0.1:49373_solr     ] o.a.s.c.SolrCore [collection1_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@170b9ba2
   [junit4]   2> 156934 INFO  (coreCloseExecutor-718-thread-4) [n:127.0.0.1:49373_solr     ] o.a.s.c.SolrCore [collection1_shard2_replica_n7]  CLOSING SolrCore org.apache.solr.core.SolrCore@6011582e
   [junit4]   2> 156934 INFO  (zkCallback-671-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 156934 INFO  (zkCallback-671-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change

[...truncated too long message...]

s.reporters.SolrJmxReporter@6fc24d15: rootName = solr_49497, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@730839a9
   [junit4]   2> 173953 INFO  (jetty-closer-1251-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 173953 INFO  (jetty-closer-1251-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@34fb1958: rootName = solr_49524, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@730839a9
   [junit4]   2> 173956 INFO  (jetty-closer-1251-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 173956 INFO  (jetty-closer-1251-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@70012f21: rootName = solr_49495, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@6e3d9813
   [junit4]   2> 173956 INFO  (jetty-closer-1251-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 173958 INFO  (jetty-closer-1251-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 173959 INFO  (jetty-closer-1251-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c33905b: rootName = solr_49497, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@6e3d9813
   [junit4]   2> 173959 INFO  (jetty-closer-1251-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 173961 INFO  (closeThreadPool-1265-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72134650694664198-127.0.0.1:49497_solr-n_0000000000) closing
   [junit4]   2> 173961 INFO  (OverseerStateUpdate-72134650694664198-127.0.0.1:49497_solr-n_0000000000) [n:127.0.0.1:49497_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:49497_solr
   [junit4]   2> 173962 INFO  (OverseerAutoScalingTriggerThread-72134650694664198-127.0.0.1:49497_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 173968 DEBUG (closeThreadPool-1265-thread-1) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 173969 INFO  (closeThreadPool-1265-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72134650694664198-127.0.0.1:49497_solr-n_0000000000) closing
   [junit4]   2> 173969 INFO  (jetty-closer-1251-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 173969 INFO  (jetty-closer-1251-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@56d6a44d: rootName = solr_49524, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@6e3d9813
   [junit4]   2> 173969 INFO  (jetty-closer-1251-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 174071 INFO  (zkCallback-1205-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 174073 INFO  (zkCallback-1241-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 174074 INFO  (jetty-closer-1251-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@710eea6a{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 174075 INFO  (jetty-closer-1251-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2fdb661e{/solr,null,UNAVAILABLE}
   [junit4]   2> 174076 INFO  (jetty-closer-1251-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 174177 INFO  (jetty-closer-1251-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72134650694664198-127.0.0.1:49497_solr-n_0000000000) closing
   [junit4]   2> 174183 INFO  (jetty-closer-1251-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@2bd39a94{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 174183 INFO  (jetty-closer-1251-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@64a0cbf4{/solr,null,UNAVAILABLE}
   [junit4]   2> 174183 INFO  (jetty-closer-1251-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 174293 INFO  (jetty-closer-1251-thread-3) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@40015472{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 174293 INFO  (jetty-closer-1251-thread-3) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6a3d9caa{/solr,null,UNAVAILABLE}
   [junit4]   2> 174293 INFO  (jetty-closer-1251-thread-3) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 174296 INFO  (SUITE-RestoreTriggerStateTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 174510 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 	4	/solr/clusterprops.json
   [junit4]   2> 	3	/solr/packages.json
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	16	/solr/live_nodes
   [junit4]   2> 	7	/solr/overseer/queue
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	4	/solr/collections
   [junit4]   2> 
   [junit4]   2> 174514 INFO  (SUITE-RestoreTriggerStateTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:49482
   [junit4]   2> 174514 INFO  (SUITE-RestoreTriggerStateTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:49482
   [junit4]   2> 174514 INFO  (SUITE-RestoreTriggerStateTest-seed#[8DFCE42228B528C4]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 49482
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.RestoreTriggerStateTest_8DFCE42228B528C4-001
   [junit4]   2> Jun 20, 2020 1:47:45 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene86, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@780cd180), locale=et-EE, timezone=Canada/Central
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 14.0.1 (64-bit)/cpus=6,threads=1,free=328314976,total=523239424
   [junit4]   2> NOTE: All tests run in this JVM: [TestDefaultStatsCache, ClusterStateTest, SparseHLLTest, HighlighterTest, CloudMLTQParserTest, BasicFunctionalityTest, ZkFailoverTest, RestoreTriggerStateTest]
   [junit4] Completed [57/910 (2!)] on J1 in 25.89s, 1 test, 1 failure <<< FAILURES!

[...truncated 47832 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6187/consoleText

[repro] Revision: 972c84022ec9c2b867848cb11f6d4c6bc466e8f2

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC"
[repro] Repro line:  ant test  -Dtestcase=ZkShardTermsTest -Dtests.seed=8DFCE42228B528C4 -Dtests.slow=true -Dtests.locale=mi -Dtests.timezone=America/Bogota -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=RestoreTriggerStateTest -Dtests.method=testEventFromRestoredState -Dtests.seed=8DFCE42228B528C4 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Canada/Central -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       RestoreTriggerStateTest
[repro]       ZkShardTermsTest
[repro] ant compile-test

[...truncated 2471 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.RestoreTriggerStateTest|*.ZkShardTermsTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Dtests.seed=8DFCE42228B528C4 -Dtests.slow=true -Dtests.locale=et-EE -Dtests.timezone=Canada/Central -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 146 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.ZkShardTermsTest
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.RestoreTriggerStateTest
[repro] Exiting with code 0

[...truncated 73 lines...]