You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/10/02 03:24:40 UTC

[JENKINS-EA] Lucene-Solr-BadApples-8.x-Linux (64bit/jdk-16-ea+15) - Build # 289 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-8.x-Linux/289/
Java: 64bit/jdk-16-ea+15 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:     1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
        at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
        at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
	at __randomizedtesting.SeedInfo.seed([9C8F6DB4252E4077]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
        at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
        at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
        at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
	at __randomizedtesting.SeedInfo.seed([9C8F6DB4252E4077]:0)




Build Log:
[...truncated 13683 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> 75896 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 75896 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/data-dir-8-001
   [junit4]   2> 75896 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 75896 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 75896 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 75897 INFO  (SUITE-BasicDistributedZkTest-seed#[9C8F6DB4252E4077]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 75900 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 75900 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 75900 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 75900 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 76000 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer start zk server on port: 32997
   [junit4]   2> 76000 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:32997
   [junit4]   2> 76000 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:32997
   [junit4]   2> 76000 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 32997
   [junit4]   2> 76001 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76005 INFO  (zkConnectionManagerCallback-642-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76005 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76013 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76017 INFO  (zkConnectionManagerCallback-644-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76017 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76018 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 76025 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 76034 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 76035 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 76036 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 76036 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 76037 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 76038 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 76038 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 76039 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 76040 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 76040 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 76126 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 76126 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 76126 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 76126 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 16-ea+15-631
   [junit4]   2> 76128 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 76128 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 76128 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 76128 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@524d919{/,null,AVAILABLE}
   [junit4]   2> 76133 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@206dcce5{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:35465}
   [junit4]   2> 76133 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.s.Server Started @76141ms
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/tempDir-001/control/data, hostPort=35465, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/control-001/cores, replicaType=NRT}
   [junit4]   2> 76134 ERROR (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.7.0
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr
   [junit4]   2> 76134 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-10-02T03:00:36.972826266Z
   [junit4]   2> 76135 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76139 INFO  (zkConnectionManagerCallback-646-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76139 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76240 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 76240 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/control-001/solr.xml
   [junit4]   2> 76247 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 76247 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 76248 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 76331 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 76332 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2d22ecad[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 76332 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2d22ecad[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 76337 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@78ca7840[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 76337 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@78ca7840[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 76338 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32997/solr
   [junit4]   2> 76339 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76340 INFO  (zkConnectionManagerCallback-656-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76340 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76444 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76446 INFO  (zkConnectionManagerCallback-658-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76446 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76494 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35465_
   [junit4]   2> 76494 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.Overseer Overseer (id=72111188048740356-127.0.0.1:35465_-n_0000000000) starting
   [junit4]   2> 76501 INFO  (OverseerStateUpdate-72111188048740356-127.0.0.1:35465_-n_0000000000) [n:127.0.0.1:35465_     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:35465_
   [junit4]   2> 76505 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35465_
   [junit4]   2> 76506 INFO  (zkCallback-657-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 76508 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 76508 WARN  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 76525 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 76540 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 76548 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 76548 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 76550 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [n:127.0.0.1:35465_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/control-001/cores
   [junit4]   2> 76569 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 76569 INFO  (zkConnectionManagerCallback-675-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 76569 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 76570 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 76571 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32997/solr ready
   [junit4]   2> 76574 INFO  (qtp27445186-1305) [n:127.0.0.1:35465_     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35465_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 76581 INFO  (OverseerThreadFactory-665-thread-1-processing-n:127.0.0.1:35465_) [n:127.0.0.1:35465_     ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 76688 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_    x:control_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 76688 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_    x:control_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 77701 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.7.0
   [junit4]   2> 77712 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 77782 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 77795 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 77796 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 77796 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 77798 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=65.9619140625, floorSegmentMB=1.3955078125, forceMergeDeletesPctAllowed=19.488234248363852, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=40.83463136224428
   [junit4]   2> 77800 WARN  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 77827 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 77827 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 77828 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 77828 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 77828 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=19, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7839481961019422]
   [junit4]   2> 77829 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 77830 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 77830 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 77830 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1679407331323936768
   [junit4]   2> 77832 INFO  (searcherExecutor-677-thread-1-processing-n:127.0.0.1:35465_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 77835 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 77835 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35465/control_collection_shard1_replica_n1/
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:35465/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 77837 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72111188048740356-core_node2-n_0000000000
   [junit4]   2> 77841 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35465/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 77942 INFO  (zkCallback-657-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 77942 INFO  (zkCallback-657-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 77943 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 77944 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1256
   [junit4]   2> 77946 INFO  (qtp27445186-1305) [n:127.0.0.1:35465_     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 78045 INFO  (zkCallback-657-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 78045 INFO  (zkCallback-657-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 78045 INFO  (zkCallback-657-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 78045 INFO  (qtp27445186-1305) [n:127.0.0.1:35465_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35465_&wt=javabin&version=2} status=0 QTime=1471
   [junit4]   2> 78046 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection
   [junit4]   2> 78153 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78153 INFO  (zkConnectionManagerCallback-686-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78153 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78162 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 78163 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32997/solr ready
   [junit4]   2> 78163 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 78164 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 78166 INFO  (OverseerThreadFactory-665-thread-2-processing-n:127.0.0.1:35465_) [n:127.0.0.1:35465_     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 78166 INFO  (OverseerCollectionConfigSetProcessor-72111188048740356-127.0.0.1:35465_-n_0000000000) [n:127.0.0.1:35465_     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 78368 WARN  (OverseerThreadFactory-665-thread-2-processing-n:127.0.0.1:35465_) [n:127.0.0.1:35465_     ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 78369 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 78370 INFO  (qtp27445186-1310) [n:127.0.0.1:35465_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=205
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 2 expected: 2
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 78373 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=4
   [junit4]   2> 78462 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-1-001 of type NRT for shard2
   [junit4]   2> 78464 WARN  (closeThreadPool-687-thread-1) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 78464 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 78464 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 78464 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 16-ea+15-631
   [junit4]   2> 78466 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 78466 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 78466 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 78467 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@124df5f3{/,null,AVAILABLE}
   [junit4]   2> 78475 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@44298780{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36929}
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.e.j.s.Server Started @78483ms
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/tempDir-001/jetty1, hostPort=36929, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-1-001/cores}
   [junit4]   2> 78476 ERROR (closeThreadPool-687-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.7.0
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr
   [junit4]   2> 78476 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-10-02T03:00:39.314658211Z
   [junit4]   2> 78477 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78481 INFO  (zkConnectionManagerCallback-689-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78481 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78554 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-2-001 of type NRT for shard1
   [junit4]   2> 78568 WARN  (closeThreadPool-687-thread-2) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 78568 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 78568 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 78569 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 16-ea+15-631
   [junit4]   2> 78576 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 78576 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 78576 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 78577 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2eef13d9{/,null,AVAILABLE}
   [junit4]   2> 78582 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 78582 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-1-001/solr.xml
   [junit4]   2> 78588 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 78588 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 78590 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 78603 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@5d28d3be{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46249}
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.e.j.s.Server Started @78611ms
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/tempDir-001/jetty2, hostPort=46249, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-2-001/cores}
   [junit4]   2> 78604 ERROR (closeThreadPool-687-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.7.0
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr
   [junit4]   2> 78604 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-10-02T03:00:39.442579592Z
   [junit4]   2> 78606 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78608 INFO  (zkConnectionManagerCallback-694-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78608 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78611 WARN  (closeThreadPool-687-thread-2-SendThread(127.0.0.1:32997)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x10030be53fa0008.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x10030be53fa0008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x10030be53fa0008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 78655 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-3-001 of type NRT for shard2
   [junit4]   2> 78656 WARN  (closeThreadPool-687-thread-3) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 78656 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 78656 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 78656 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 16-ea+15-631
   [junit4]   2> 78688 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 78688 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 78688 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 78688 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cc6ad26{/,null,AVAILABLE}
   [junit4]   2> 78707 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.AbstractConnector Started ServerConnector@1444bf4{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:33265}
   [junit4]   2> 78707 INFO  (closeThreadPool-687-thread-3) [     ] o.e.j.s.Server Started @78715ms
   [junit4]   2> 78707 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/tempDir-001/jetty3, hostPort=33265, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-3-001/cores}
   [junit4]   2> 78708 ERROR (closeThreadPool-687-thread-3) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 78708 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 78708 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.7.0
   [junit4]   2> 78708 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 78708 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr
   [junit4]   2> 78708 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-10-02T03:00:39.546305978Z
   [junit4]   2> 78712 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 78712 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-2-001/solr.xml
   [junit4]   2> 78715 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 78715 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 78716 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 78721 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78730 INFO  (zkConnectionManagerCallback-696-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78730 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78739 INFO  (TEST-BasicDistributedZkTest.test-seed#[9C8F6DB4252E4077]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-4-001 of type NRT for shard1
   [junit4]   2> 78740 WARN  (closeThreadPool-687-thread-4) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 78740 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 78740 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 78740 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 16-ea+15-631
   [junit4]   2> 78779 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 78799 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 78799 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 78799 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 78800 WARN  (closeThreadPool-687-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3794811[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78800 WARN  (closeThreadPool-687-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3794811[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78803 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72881f51{/,null,AVAILABLE}
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.AbstractConnector Started ServerConnector@77a9e551{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:43117}
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.e.j.s.Server Started @78818ms
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/tempDir-001/jetty4, hostPort=43117, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-4-001/cores}
   [junit4]   2> 78810 ERROR (closeThreadPool-687-thread-4) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.7.0
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr
   [junit4]   2> 78810 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-10-02T03:00:39.648773613Z
   [junit4]   2> 78812 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78812 WARN  (closeThreadPool-687-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3892d2f4[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78812 WARN  (closeThreadPool-687-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3892d2f4[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78813 INFO  (zkConnectionManagerCallback-704-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78813 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78813 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32997/solr
   [junit4]   2> 78815 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78817 INFO  (zkConnectionManagerCallback-708-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78817 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78831 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 78831 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-3-001/solr.xml
   [junit4]   2> 78834 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 78834 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 78835 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 78881 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 78882 WARN  (closeThreadPool-687-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@45e3cdd9[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78882 WARN  (closeThreadPool-687-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@45e3cdd9[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78887 WARN  (closeThreadPool-687-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@becd7a1[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78887 WARN  (closeThreadPool-687-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@becd7a1[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78888 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32997/solr
   [junit4]   2> 78893 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78909 INFO  (zkConnectionManagerCallback-718-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78909 INFO  (closeThreadPool-687-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 78920 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 78921 WARN  (closeThreadPool-687-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3d8f356c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78921 WARN  (closeThreadPool-687-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3d8f356c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78930 WARN  (closeThreadPool-687-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5dc5b8df[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78930 WARN  (closeThreadPool-687-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5dc5b8df[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 78931 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32997/solr
   [junit4]   2> 78931 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 78931 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-4-001/solr.xml
   [junit4]   2> 78932 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 78935 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 78935 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 78936 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 78937 INFO  (zkConnectionManagerCallback-725-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 78937 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79033 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 79037 INFO  (zkConnectionManagerCallback-730-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 79037 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79042 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 79050 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.ZkController Publish node=127.0.0.1:36929_ as DOWN
   [junit4]   2> 79053 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 79053 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36929_
   [junit4]   2> 79056 INFO  (zkCallback-685-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 79056 INFO  (zkCallback-657-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 79056 INFO  (zkCallback-729-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 79057 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 79057 WARN  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 79062 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 79066 WARN  (closeThreadPool-687-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2a9d9a38[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 79066 WARN  (closeThreadPool-687-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2a9d9a38[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 79081 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 79084 WARN  (closeThreadPool-687-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7e13b57d[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 79084 WARN  (closeThreadPool-687-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7e13b57d[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 79086 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32997/solr
   [junit4]   2> 79093 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 79094 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79101 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79101 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79102 INFO  (closeThreadPool-687-thread-1) [n:127.0.0.1:36929_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-1-001/cores
   [junit4]   2> 79117 INFO  (zkConnectionManagerCallback-744-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 79117 INFO  (closeThreadPool-687-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79118 WARN  (closeThreadPool-687-thread-4-SendThread(127.0.0.1:32997)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x10030be53fa000f.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x10030be53fa000f, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x10030be53fa000f, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 79145 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 79153 INFO  (zkConnectionManagerCallback-747-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 79153 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79224 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 79227 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.ZkController Publish node=127.0.0.1:33265_ as DOWN
   [junit4]   2> 79228 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 79228 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33265_
   [junit4]   2> 79228 INFO  (closeThreadPool-687-thread-1) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:36929_
   [junit4]   2> 79229 INFO  (zkCallback-657-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 79229 INFO  (zkCallback-685-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 79230 INFO  (zkCallback-729-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 79231 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 79231 WARN  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 79237 INFO  (zkCallback-746-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 79249 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 79253 INFO  (zkConnectionManagerCallback-749-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 79253 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79256 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 79259 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 79261 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.ZkController Publish node=127.0.0.1:43117_ as DOWN
   [junit4]   2> 79261 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 79262 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 79262 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43117_
   [junit4]   2> 79262 INFO  (zkConnectionManagerCallback-754-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 79262 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 79263 INFO  (zkCallback-657-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 79263 INFO  (zkCallback-685-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 79263 INFO  (zkCallback-729-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 79263 INFO  (zkCallback-746-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 79263 INFO  (zkCallback-748-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 79264 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 79264 WARN  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 79264 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 79268 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.ZkController Publish node=127.0.0.1:46249_ as DOWN
   [junit4]   2> 79268 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 79268 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46249_
   [junit4]   2> 79270 INFO  (zkCallback-657-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79270 INFO  (zkCallback-746-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79270 INFO  (zkCallback-729-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79270 INFO  (zkCallback-685-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79270 INFO  (zkCallback-748-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79271 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 79271 WARN  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 79276 INFO  (zkCallback-753-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 79277 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79283 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79284 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79285 INFO  (closeThreadPool-687-thread-3) [n:127.0.0.1:33265_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-3-001/cores
   [junit4]   2> 79293 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 79293 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 79299 INFO  (closeThreadPool-687-thread-3) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:33265_
   [junit4]   2> 79297 WARN  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.m.r.j.JmxMetricsReporter Unable to register meter
   [junit4]   2>           => javax.management.InstanceNotFoundException: solr:dom1=node,category=QUERY,scope=/admin/metrics/history,name=timeouts
   [junit4]   2> 	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083)
   [junit4]   2> javax.management.InstanceNotFoundException: solr:dom1=node,category=QUERY,scope=/admin/metrics/history,name=timeouts
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:423) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:411) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[?:?]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:534) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onMeterAdded(JmxMetricsReporter.java:655) [java/:?]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:533) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.addListener(MetricRegistry.java:368) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:736) [java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:109) [java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70) [java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:983) [java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:911) [java/:?]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:761) [java/:?]
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:275) [java/:?]
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:184) [java/:?]
   [junit4]   2> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:134) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:751) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) [?:?]
   [junit4]   2> 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) [?:?]
   [junit4]   2> 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) [?:?]
   [junit4]   2> 	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:755) [?:?]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1448) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1507) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1164) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1003) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:459) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:382) [java/:?]
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:193) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:566) [java/:?]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:504) [java/:?]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:472) [java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.lambda$createJettys$1(AbstractFullDistribZkTestBase.java:484) [java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:218) [java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:832) [?:?]
   [junit4]   2> 79312 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79314 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79322 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79322 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79323 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79323 INFO  (closeThreadPool-687-thread-2) [n:127.0.0.1:46249_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-2-001/cores
   [junit4]   2> 79323 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b4d602f
   [junit4]   2> 79324 INFO  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-8.x-Linux/solr/build/solr-core/test/J5/temp/solr.cloud.BasicDistributedZkTest_9C8F6DB4252E4077-001/shard-4-001/cores
   [junit4]   2> 79326 WARN  (closeThreadPool-687-thread-4) [n:127.0.0.1:43117_     ] o.a.s.m.r.j.JmxMetricsReporter Unable to register meter
   [junit4]   2>           => javax.management.InstanceNotFoundException: solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=errors
   [junit4]   2> 	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083)
   [junit4]   2> javax.management.InstanceNotFoundException: solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=errors
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:423) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:411) ~[?:?]
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[?:?]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:534) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onMeterAdded(JmxMetricsReporter.java:655) [java/:?]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:533) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:521) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:154) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:496) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at com.codahale.metrics.MetricRegistry.meter(MetricRegistry.java:258) [metrics-core-4.1.5.jar:4.1.5]
   [junit4]   2> 	at org.apache.solr.metrics.SolrMetricManager.meter(SolrMetricManager.java:633) [java/:?]
   [junit4]   2> 	at org.apache.solr.metrics.SolrMetricsContext.meter(SolrMetricsContext.java:76) [java/:?]
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:157) [java/:?]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:905) [java/:?]
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:275) [java/:?]
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:184) [java/:?]
   [junit4]   2> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:134) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:751) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) [?:?]
   [junit4]   2> 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) [?:?]
   [junit4]   2> 	at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) [?:?]
   [junit4]   2> 	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:755) [?:?]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1448) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1507) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1164) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1003) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:459) [jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:382) [java/:?]
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:193) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:566) [java/:?]
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySol

[...truncated too long message...]

 Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]   2>         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]   2>         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
   [junit4]   2> Oct 02, 2020 3:01:40 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]   2> Oct 02, 2020 3:01:43 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]   2>         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]   2>         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]   2>         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {regex_dup_A_s=PostingsFormat(name=LuceneVarGapFixedInterval), regex_dup_B_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), SubjectTerms_mfacet=PostingsFormat(name=LuceneVarGapDocFreqInterval), multiDefault=FST50, _root_=PostingsFormat(name=LuceneVarGapDocFreqInterval), genre_s=PostingsFormat(name=LuceneVarGapFixedInterval), author_t=FST50, series_t=FST50, rnd_b=Lucene84, oddField_s=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), cat=PostingsFormat(name=LuceneVarGapDocFreqInterval), foo_b=PostingsFormat(name=LuceneVarGapFixedInterval), name=Lucene84, inStock=PostingsFormat(name=LuceneVarGapFixedInterval), id=Lucene84, text=PostingsFormat(name=LuceneVarGapFixedInterval), t_sortable=Lucene84}, docValues:{other_tl1=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Lucene80), n_l1=DocValuesFormat(name=Direct), intDefault=DocValuesFormat(name=Direct), n_td1=DocValuesFormat(name=Lucene80), n_d1=DocValuesFormat(name=Direct), range_facet_l=DocValuesFormat(name=Direct), n_f1=DocValuesFormat(name=Lucene80), n_tl1=DocValuesFormat(name=Lucene80), n_tf1=DocValuesFormat(name=Direct), price=DocValuesFormat(name=Asserting), sequence_i=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Asserting), t_sortable=DocValuesFormat(name=Lucene80), timestamp=DocValuesFormat(name=Direct), foo_i=DocValuesFormat(name=Lucene80), val_i=DocValuesFormat(name=Lucene80), n_dt1=DocValuesFormat(name=Lucene80), a_i1=DocValuesFormat(name=Direct), n_ti1=DocValuesFormat(name=Lucene80), _version_=DocValuesFormat(name=Direct), n_tdt1=DocValuesFormat(name=Direct), id_i1=DocValuesFormat(name=Lucene80), foo_d=DocValuesFormat(name=Lucene80), range_facet_i_dv=DocValuesFormat(name=Direct), foo_f=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=630, maxMBSortInHeap=6.853370650369532, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=nl-SR, timezone=Etc/GMT+3
   [junit4]   2> NOTE: Linux 5.4.0-48-generic amd64/Oracle Corporation 16-ea (64-bit)/cpus=16,threads=2,free=72029432,total=502792192
   [junit4]   2> NOTE: All tests run in this JVM: [SyncSliceTest, TestTrackingShardHandlerFactory, TestReplicationHandlerDiskOverFlow, SortSpecParsingTest, DeleteShardTest, AnalysisErrorHandlingTest, TokenizerChainTest, BasicDistributedZkTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=9C8F6DB4252E4077 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-SR -Dtests.timezone=Etc/GMT+3 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J5 | BasicDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   [junit4]    >    1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]    >         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]    >         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C8F6DB4252E4077]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=1982, name=aliveCheckExecutor-1076-thread-1, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]    >         at java.base@16-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1661)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1056)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]    >         at java.base@16-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]    >         at java.base@16-ea/java.lang.Thread.run(Thread.java:832)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C8F6DB4252E4077]:0)
   [junit4] Completed [78/920 (1!)] on J5 in 66.34s, 1 test, 2 errors <<< FAILURES!

[...truncated 47667 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-8.x-Linux/289/consoleText

[repro] Revision: 587f7302b9de62f840d85ca325a660a9fbe1e3b0

[repro] Repro line:  ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=9C8F6DB4252E4077 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-SR -Dtests.timezone=Etc/GMT+3 -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       BasicDistributedZkTest
[repro] ant compile-test

[...truncated 3454 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.BasicDistributedZkTest" -Dtests.showOutput=onerror  -Dtests.seed=9C8F6DB4252E4077 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-SR -Dtests.timezone=Etc/GMT+3 -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 81 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.BasicDistributedZkTest
[repro] Exiting with code 0

[...truncated 63 lines...]