You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/06/22 21:44:54 UTC

[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-14.0.1) - Build # 1173 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/1173/
Java: 64bit/jdk-14.0.1 -XX:-UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([1D50F113160491FD]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([1D50F113160491FD]:0)




Build Log:
[...truncated 14945 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> 1468766 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1468766 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1468766 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/data-dir-107-001
   [junit4]   2> 1468766 WARN  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
   [junit4]   2> 1468766 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1468767 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1468780 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.SolrTestCaseJ4 ###Starting deleteReplicaOnIndexing
   [junit4]   2> 1468780 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001
   [junit4]   2> 1468780 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1468781 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1468781 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1468888 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer start zk server on port: 64387
   [junit4]   2> 1468888 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:64387
   [junit4]   2> 1468888 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:64387
   [junit4]   2> 1468888 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 64387
   [junit4]   2> 1468891 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1468903 INFO  (zkConnectionManagerCallback-10281-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1468903 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1468908 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1468912 INFO  (zkConnectionManagerCallback-10283-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1468912 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1468913 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1468916 INFO  (zkConnectionManagerCallback-10285-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1468916 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469028 WARN  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 1469028 WARN  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1469028 WARN  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1469028 WARN  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1469028 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 1469033 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1469033 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1469033 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1469034 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f01d3b3{/solr,null,AVAILABLE}
   [junit4]   2> 1469035 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@1fa3c0db{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:64396}
   [junit4]   2> 1469035 INFO  (jetty-launcher-10286-thread-1) [     ] o.e.j.s.Server Started @1469216ms
   [junit4]   2> 1469035 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64396}
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1469036 ERROR (jetty-launcher-10286-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.6.0
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-22T21:00:51.694231Z
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1469036 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@775dff3e{/solr,null,AVAILABLE}
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.AbstractConnector Started ServerConnector@1b97916b{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:64397}
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.e.j.s.Server Started @1469218ms
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64397}
   [junit4]   2> 1469037 ERROR (jetty-launcher-10286-thread-3) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.6.0
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr
   [junit4]   2> 1469037 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-22T21:00:51.695949Z
   [junit4]   2> 1469039 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1469039 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1469039 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1469040 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ff2a049{/solr,null,AVAILABLE}
   [junit4]   2> 1469040 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6fcd2ef9{/solr,null,AVAILABLE}
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@5abcba6d{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:64398}
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-2) [     ] o.e.j.s.Server Started @1469222ms
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64398}
   [junit4]   2> 1469041 ERROR (jetty-launcher-10286-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1469041 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.6.0
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-22T21:00:51.700067Z
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.AbstractConnector Started ServerConnector@28faf7cf{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:64399}
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-4) [     ] o.e.j.s.Server Started @1469223ms
   [junit4]   2> 1469042 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64399}
   [junit4]   2> 1469043 ERROR (jetty-launcher-10286-thread-4) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1469043 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1469043 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.6.0
   [junit4]   2> 1469043 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1469043 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr
   [junit4]   2> 1469043 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-06-22T21:00:51.701179Z
   [junit4]   2> 1469045 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469045 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469046 INFO  (zkConnectionManagerCallback-10288-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469046 INFO  (zkConnectionManagerCallback-10290-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469046 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469046 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469048 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1469048 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1469049 INFO  (zkConnectionManagerCallback-10292-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469050 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469050 INFO  (zkConnectionManagerCallback-10294-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469050 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469051 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1469057 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1469327 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1469336 WARN  (jetty-launcher-10286-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@76e1f6ae[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469336 WARN  (jetty-launcher-10286-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@76e1f6ae[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469347 WARN  (jetty-launcher-10286-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@c04e417[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469348 WARN  (jetty-launcher-10286-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@c04e417[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469349 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64387/solr
   [junit4]   2> 1469351 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469356 INFO  (zkConnectionManagerCallback-10311-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469357 INFO  (jetty-launcher-10286-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469473 INFO  (zkConnectionManagerCallback-10316-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469474 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469474 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469516 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1469521 WARN  (jetty-launcher-10286-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1c16eb27[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469521 WARN  (jetty-launcher-10286-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1c16eb27[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469526 WARN  (jetty-launcher-10286-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3229a03e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469526 WARN  (jetty-launcher-10286-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3229a03e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469527 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64387/solr
   [junit4]   2> 1469531 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469533 INFO  (zkConnectionManagerCallback-10326-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469534 INFO  (jetty-launcher-10286-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469564 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1469570 WARN  (jetty-launcher-10286-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@287fa324[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469570 WARN  (jetty-launcher-10286-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@287fa324[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469577 WARN  (jetty-launcher-10286-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5913ed77[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469577 WARN  (jetty-launcher-10286-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5913ed77[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1469579 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64387/solr
   [junit4]   2> 1469582 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469588 INFO  (zkConnectionManagerCallback-10334-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469588 INFO  (jetty-launcher-10286-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469656 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469661 INFO  (zkConnectionManagerCallback-10336-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469663 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469705 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469708 INFO  (zkConnectionManagerCallback-10338-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1469708 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1469818 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64399_solr
   [junit4]   2> 1469828 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.Overseer Overseer (id=72150511124480011-127.0.0.1:64399_solr-n_0000000000) starting
   [junit4]   2> 1469838 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64398_solr
   [junit4]   2> 1469857 INFO  (zkCallback-10337-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1469861 INFO  (zkCallback-10335-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1469909 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1469915 WARN  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1469938 INFO  (OverseerStateUpdate-72150511124480011-127.0.0.1:64399_solr-n_0000000000) [n:127.0.0.1:64399_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:64399_solr
   [junit4]   2> 1469957 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:64399_solr as DOWN
   [junit4]   2> 1469984 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1470006 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1470006 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64399_solr
   [junit4]   2> 1470035 INFO  (zkCallback-10335-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1470035 INFO  (zkCallback-10337-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1470054 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1470063 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1470064 WARN  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1470078 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64398.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470080 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1470087 WARN  (jetty-launcher-10286-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1e9db138[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1470087 WARN  (jetty-launcher-10286-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1e9db138[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1470095 WARN  (jetty-launcher-10286-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2233c5b1[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1470095 WARN  (jetty-launcher-10286-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2233c5b1[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1470097 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64387/solr
   [junit4]   2> 1470100 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470103 INFO  (zkConnectionManagerCallback-10364-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1470104 INFO  (jetty-launcher-10286-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470118 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64399.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470119 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64398.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470119 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64398.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470131 INFO  (jetty-launcher-10286-thread-2) [n:127.0.0.1:64398_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node2
   [junit4]   2> 1470137 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64399.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470137 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64399.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470141 INFO  (jetty-launcher-10286-thread-4) [n:127.0.0.1:64399_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node4
   [junit4]   2> 1470232 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470242 INFO  (zkConnectionManagerCallback-10368-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1470243 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470304 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1470364 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:64396_solr as DOWN
   [junit4]   2> 1470372 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1470372 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64396_solr
   [junit4]   2> 1470378 INFO  (zkCallback-10337-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1470379 INFO  (zkCallback-10367-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1470379 INFO  (zkCallback-10335-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1470393 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1470394 WARN  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1470415 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1470422 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1470456 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:64397_solr as DOWN
   [junit4]   2> 1470462 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1470462 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64397_solr
   [junit4]   2> 1470467 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64396.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470471 INFO  (zkCallback-10335-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1470471 INFO  (zkCallback-10337-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1470471 INFO  (zkCallback-10367-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1470471 INFO  (zkCallback-10315-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1470481 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64396.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470481 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64396.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470482 INFO  (jetty-launcher-10286-thread-1) [n:127.0.0.1:64396_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node1
   [junit4]   2> 1470484 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1470485 WARN  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1470502 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1470527 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64397.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470536 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64397.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470536 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64397.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1470537 INFO  (jetty-launcher-10286-thread-3) [n:127.0.0.1:64397_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node3
   [junit4]   2> 1470550 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=4
   [junit4]   2> 1470551 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1470557 INFO  (zkConnectionManagerCallback-10387-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1470557 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1470560 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1470563 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64387/solr ready
   [junit4]   2> 1470573 INFO  (qtp1417371282-15808) [n:127.0.0.1:64398_solr     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=deleteReplicaOnIndexing&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1470580 INFO  (OverseerThreadFactory-10349-thread-1-processing-n:127.0.0.1:64399_solr) [n:127.0.0.1:64399_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection deleteReplicaOnIndexing
   [junit4]   2> 1470699 INFO  (qtp1417371282-15812) [n:127.0.0.1:64398_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1470700 INFO  (qtp1417371282-15812) [n:127.0.0.1:64398_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1470702 INFO  (qtp291781767-15799) [n:127.0.0.1:64396_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1470705 INFO  (qtp357596593-15804) [n:127.0.0.1:64397_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1470708 INFO  (qtp641539448-15810) [n:127.0.0.1:64399_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1470773 INFO  (OverseerStateUpdate-72150511124480011-127.0.0.1:64399_solr-n_0000000000) [n:127.0.0.1:64399_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteReplicaOnIndexing",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"deleteReplicaOnIndexing_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64399/solr",
   [junit4]   2>   "node_name":"127.0.0.1:64399_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1470781 INFO  (OverseerStateUpdate-72150511124480011-127.0.0.1:64399_solr-n_0000000000) [n:127.0.0.1:64399_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteReplicaOnIndexing",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"deleteReplicaOnIndexing_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64397/solr",
   [junit4]   2>   "node_name":"127.0.0.1:64397_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1470999 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr    x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteReplicaOnIndexing_shard1_replica_n1&action=CREATE&numShards=1&collection=deleteReplicaOnIndexing&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1471006 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr    x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteReplicaOnIndexing_shard1_replica_n2&action=CREATE&numShards=1&collection=deleteReplicaOnIndexing&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1471037 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.6.0
   [junit4]   2> 1471043 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.6.0
   [junit4]   2> 1471051 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1471052 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1471053 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1471053 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'deleteReplicaOnIndexing_shard1_replica_n2' using configuration from configset conf, trusted=true
   [junit4]   2> 1471053 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64397.solr.core.deleteReplicaOnIndexing.shard1.replica_n2' (registry 'solr.core.deleteReplicaOnIndexing.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1471054 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.SolrCore [[deleteReplicaOnIndexing_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node3/deleteReplicaOnIndexing_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node3/deleteReplicaOnIndexing_shard1_replica_n2/data/]
   [junit4]   2> 1471054 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1471054 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'deleteReplicaOnIndexing_shard1_replica_n1' using configuration from configset conf, trusted=true
   [junit4]   2> 1471055 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64399.solr.core.deleteReplicaOnIndexing.shard1.replica_n1' (registry 'solr.core.deleteReplicaOnIndexing.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea9e5d4
   [junit4]   2> 1471055 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.SolrCore [[deleteReplicaOnIndexing_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node4/deleteReplicaOnIndexing_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest_1D50F113160491FD-001/tempDir-001/node4/deleteReplicaOnIndexing_shard1_replica_n1/data/]
   [junit4]   2> 1471138 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1471138 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1471155 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1471155 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1471171 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1471179 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1471179 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1471181 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1471182 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1471182 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1670234404384931840
   [junit4]   2> 1471183 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1471183 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1471191 INFO  (searcherExecutor-10389-thread-1-processing-n:127.0.0.1:64397_solr x:deleteReplicaOnIndexing_shard1_replica_n2 c:deleteReplicaOnIndexing s:shard1 r:core_node4) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.SolrCore [deleteReplicaOnIndexing_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1471197 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1471199 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1471199 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1471199 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1670234404402757632
   [junit4]   2> 1471211 INFO  (searcherExecutor-10391-thread-1-processing-n:127.0.0.1:64399_solr x:deleteReplicaOnIndexing_shard1_replica_n1 c:deleteReplicaOnIndexing s:shard1 r:core_node3) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.SolrCore [deleteReplicaOnIndexing_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1471210 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteReplicaOnIndexing/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1471211 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/deleteReplicaOnIndexing/leaders/shard1
   [junit4]   2> 1471229 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteReplicaOnIndexing/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 1471229 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/deleteReplicaOnIndexing/leaders/shard1
   [junit4]   2> 1471260 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1471260 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1471261 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/
   [junit4]   2> 1471262 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.PeerSync PeerSync: core=deleteReplicaOnIndexing_shard1_replica_n2 url=http://127.0.0.1:64397/solr  START replicas=[http://127.0.0.1:64399/solr/deleteReplicaOnIndexing_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1471263 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.PeerSync PeerSync: core=deleteReplicaOnIndexing_shard1_replica_n2 url=http://127.0.0.1:64397/solr  DONE. We have no versions. sync failed.
   [junit4]   2> 1471267 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.c.S.Request [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1471270 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1471270 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1471270 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/deleteReplicaOnIndexing/leaders/shard1/leader after winning as /collections/deleteReplicaOnIndexing/leader_elect/shard1/election/72150511124480008-core_node4-n_0000000000
   [junit4]   2> 1471285 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/ shard1
   [junit4]   2> 1471391 INFO  (zkCallback-10315-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1471391 INFO  (zkCallback-10315-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1471403 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1471409 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteReplicaOnIndexing_shard1_replica_n2&action=CREATE&numShards=1&collection=deleteReplicaOnIndexing&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=403
   [junit4]   2> 1471511 INFO  (zkCallback-10315-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1471511 INFO  (zkCallback-10315-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1471511 INFO  (zkCallback-10315-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472267 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteReplicaOnIndexing_shard1_replica_n1&action=CREATE&numShards=1&collection=deleteReplicaOnIndexing&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1267
   [junit4]   2> 1472272 INFO  (qtp1417371282-15808) [n:127.0.0.1:64398_solr     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1472371 INFO  (zkCallback-10335-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472371 INFO  (zkCallback-10337-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472371 INFO  (zkCallback-10315-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472371 INFO  (zkCallback-10315-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472371 INFO  (zkCallback-10315-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472371 INFO  (zkCallback-10335-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteReplicaOnIndexing/state.json] for collection [deleteReplicaOnIndexing] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1472373 INFO  (qtp1417371282-15808) [n:127.0.0.1:64398_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=deleteReplicaOnIndexing&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=1800
   [junit4]   2> 1472374 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.SolrCloudTestCase waitForState (deleteReplicaOnIndexing): 
   [junit4]   2> 1472376 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 2 expected replica count: 2
   [junit4]   2> 1472377 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 2 expected replica count: 2
   [junit4]   2> 1472378 INFO  (TEST-DeleteReplicaTest.deleteReplicaOnIndexing-seed#[1D50F113160491FD]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 2 expected replica count: 2
   [junit4]   2> 1472399 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deleteReplicaOnIndexing/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
   [junit4]   2> 1472435 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[130000 (1670234405661048832)]} 0 5
   [junit4]   2> 1472435 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[30000 (1670234405651611648)]} 0 0
   [junit4]   2> 1472436 INFO  (qtp357596593-15965) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130000 (1670234405661048832)]} 0 36
   [junit4]   2> 1472436 INFO  (qtp357596593-15797) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30000 (1670234405651611648)]} 0 52
   [junit4]   2> 1472438 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[80000 (1670234405652660224)]} 0 0
   [junit4]   2> 1472438 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[160000 (1670234405669437440)]} 0 0
   [junit4]   2> 1472439 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[90000 (1670234405652660226)]} 0 0
   [junit4]   2> 1472439 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[70000 (1670234405652660225)]} 0 0
   [junit4]   2> 1472439 INFO  (qtp357596593-15947) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80000 (1670234405652660224)]} 0 48
   [junit4]   2> 1472439 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[50000 (1670234405652660227)]} 0 0
   [junit4]   2> 1472439 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[180000 (1670234405675728896)]} 0 0
   [junit4]   2> 1472440 INFO  (qtp357596593-15983) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160000 (1670234405669437440)]} 0 32
   [junit4]   2> 1472440 INFO  (qtp357596593-15948) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90000 (1670234405652660226)]} 0 48
   [junit4]   2> 1472440 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[190000 (1670234405676777472)]} 0 0
   [junit4]   2> 1472441 INFO  (qtp357596593-15804) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50000 (1670234405652660227)]} 0 56
   [junit4]   2> 1472441 INFO  (qtp357596593-15988) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180000 (1670234405675728896)]} 0 28
   [junit4]   2> 1472441 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[100000 (1670234405657903104)]} 0 0
   [junit4]   2> 1472441 INFO  (qtp357596593-15946) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70000 (1670234405652660225)]} 0 49
   [junit4]   2> 1472441 INFO  (qtp357596593-15998) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190000 (1670234405676777472)]} 0 27
   [junit4]   2> 1472442 INFO  (qtp357596593-15953) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100000 (1670234405657903104)]} 0 45
   [junit4]   2> 1472446 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[120000 (1670234405660000256)]} 0 0
   [junit4]   2> 1472446 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[250000 (1670234405699846144)]} 0 0
   [junit4]   2> 1472446 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[260000 (1670234405695651840)]} 0 0
   [junit4]   2> 1472446 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[60000 (1670234405653708800)]} 0 17
   [junit4]   2> 1472447 INFO  (qtp357596593-15963) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120000 (1670234405660000256)]} 0 49
   [junit4]   2> 1472447 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60000 (1670234405653708800)]} 0 59
   [junit4]   2> 1472447 INFO  (qtp357596593-16034) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[250000 (1670234405699846144)]} 0 11
   [junit4]   2> 1472447 INFO  (qtp357596593-16025) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[260000 (1670234405695651840)]} 0 15
   [junit4]   2> 1472449 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[300000 (1670234405702991872)]} 0 0
   [junit4]   2> 1472450 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[290000 (1670234405702991874)]} 0 0
   [junit4]   2> 1472450 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[30001 (1670234405701943296)]} 0 0
   [junit4]   2> 1472450 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[200000 (1670234405684117505)]} 0 0
   [junit4]   2> 1472450 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[80001 (1670234405705089024)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp357596593-15947) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[290000 (1670234405702991874)]} 0 11
   [junit4]   2> 1472451 INFO  (qtp357596593-16005) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[200000 (1670234405684117505)]} 0 30
   [junit4]   2> 1472451 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[70001 (1670234405707186177)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[180001 (1670234405707186178)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[160001 (1670234405707186176)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[270000 (1670234405707186179)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp357596593-16037) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30001 (1670234405701943296)]} 0 13
   [junit4]   2> 1472451 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[130001 (1670234405702991873)]} 0 0
   [junit4]   2> 1472451 INFO  (qtp357596593-15965) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[300000 (1670234405702991872)]} 0 12
   [junit4]   2> 1472452 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[260001 (1670234405713477632)]} 0 0
   [junit4]   2> 1472452 INFO  (qtp641539448-16055) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[310000 (1670234405711380480)]} 0 0
   [junit4]   2> 1472453 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[60001 (1670234405713477633)]} 0 0
   [junit4]   2> 1472453 INFO  (qtp641539448-16055) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[250001 (1670234405715574784)]} 0 0
   [junit4]   2> 1472454 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[50001 (1670234405716623361)]} 0 0
   [junit4]   2> 1472454 INFO  (qtp357596593-15983) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80001 (1670234405705089024)]} 0 12
   [junit4]   2> 1472454 INFO  (qtp357596593-15804) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70001 (1670234405707186177)]} 0 11
   [junit4]   2> 1472454 INFO  (qtp357596593-15988) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[270000 (1670234405707186179)]} 0 10
   [junit4]   2> 1472454 INFO  (qtp357596593-16038) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180001 (1670234405707186178)]} 0 11
   [junit4]   2> 1472454 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[320000 (1670234405716623360)]} 0 0
   [junit4]   2> 1472454 INFO  (qtp357596593-15797) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130001 (1670234405702991873)]} 0 15
   [junit4]   2> 1472454 INFO  (qtp357596593-15953) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[310000 (1670234405711380480)]} 0 7
   [junit4]   2> 1472455 INFO  (qtp357596593-16041) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60001 (1670234405713477633)]} 0 6
   [junit4]   2> 1472455 INFO  (qtp357596593-15947) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[250001 (1670234405715574784)]} 0 4
   [junit4]   2> 1472455 INFO  (qtp357596593-15948) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160001 (1670234405707186176)]} 0 12
   [junit4]   2> 1472455 INFO  (qtp641539448-16050) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[120001 (1670234405716623362)]} 0 0
   [junit4]   2> 1472455 INFO  (qtp641539448-16055) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[90001 (1670234405715574786)]} 0 1
   [junit4]   2> 1472455 INFO  (qtp641539448-16049) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[190001 (1670234405715574785)]} 0 1
   [junit4]   2> 1472455 INFO  (qtp357596593-15965) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50001 (1670234405716623361)]} 0 3
   [junit4]   2> 1472455 INFO  (qtp357596593-16037) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[320000 (1670234405716623360)]} 0 3
   [junit4]   2> 1472455 INFO  (qtp357596593-15946) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[260001 (1670234405713477632)]} 0 6
   [junit4]   2> 1472455 INFO  (qtp357596593-16005) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190001 (1670234405715574785)]} 0 4
   [junit4]   2> 1472455 INFO  (qtp357596593-15805) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120001 (1670234405716623362)]} 0 3
   [junit4]   2> 1472455 INFO  (qtp357596593-15998) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90001 (1670234405715574786)]} 0 4
   [junit4]   2> 1472461 INFO  (qtp641539448-15811) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[200001 (1670234405719769088)]} 0 5
   [junit4]   2> 1472461 INFO  (qtp641539448-15807) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:64397/solr/deleteReplicaOnIndexing_shard1_replica_n2/&wt=javabin&version=2}{add=[80002 (1670234405719769089)]} 0 1
   [junit4]   2> 1472462 INFO  (qtp357596593-15983) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80002 (1670234405719769089)]} 0 6
   [junit4]   2> 1472462 INFO  (qtp357596593-15948) [n:127.0.0.1:64397_solr c:deleteReplicaOnIndexing s:shard1 r:core_node4 x:deleteReplicaOnIndexing_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [deleteReplicaOnIndexing_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[200001 (1670234405719769088)]} 0 7
   [junit4]   2> 1472468 INFO  (qtp641539448-16055) [n:127.0.0.1:64399_solr c:deleteReplicaOnIndexing s:shard1 r:core_node3 x:deleteReplicaOnIndexing_shard1_r

[...truncated too long message...]

on java.util.concurrent.SynchronousQueue$TransferStack@3680defb
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1055)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]   2> 	at java.base@14.0.1/java.lang.Thread.run(Thread.java:832)
   [junit4]   2> 
   [junit4]   2> "closeThreadPool-10711-thread-4" ID=16886 TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@3680defb
   [junit4]   2> 	at java.base@14.0.1/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]   2> 	- timed waiting on java.util.concurrent.SynchronousQueue$TransferStack@3680defb
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1055)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1116)
   [junit4]   2> 	at java.base@14.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
   [junit4]   2> 	at java.base@14.0.1/java.lang.Thread.run(Thread.java:832)
   [junit4]   2> 
   [junit4]   2> ^^==============================================
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DeleteReplicaTest -Dtests.method=deleteReplicaAndVerifyDirectoryCleanup -Dtests.seed=1D50F113160491FD -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7.85s J3 | DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1D50F113160491FD]:0)
   [junit4]   2> birž. 22, 2020 2:01:12 POPIET com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 25 leaked thread(s).
   [junit4]   2> 1489823 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/collections/deletereplica_test/terms/shard1
   [junit4]   2> 	5	/solr/aliases.json
   [junit4]   2> 	5	/solr/clusterprops.json
   [junit4]   2> 	4	/solr/packages.json
   [junit4]   2> 	4	/solr/security.json
   [junit4]   2> 	2	/solr/configs/conf
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	32	/solr/collections/deletereplica_test/state.json
   [junit4]   2> 	5	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	26	/solr/overseer/queue
   [junit4]   2> 	22	/solr/live_nodes
   [junit4]   2> 	10	/solr/collections
   [junit4]   2> 	9	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 1489835 INFO  (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:64632
   [junit4]   2> 1489835 INFO  (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:64632
   [junit4]   2> 1489836 INFO  (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[1D50F113160491FD]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 64632
   [junit4]   2> 1489836 INFO  (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[1D50F113160491FD]) [     ] o.a.s.SolrTestCaseJ4 ###Ending deleteReplicaAndVerifyDirectoryCleanup
   [junit4]   2> 1489838 INFO  (SUITE-DeleteReplicaTest-seed#[1D50F113160491FD]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {_root_=PostingsFormat(name=MockRandom), id=BlockTreeOrds(blocksize=128)}, docValues:{}, maxPointsInLeafNode=630, maxMBSortInHeap=6.750191925978602, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@67e40d34), locale=lt, timezone=US/Pacific-New
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 14.0.1 (64-bit)/cpus=6,threads=1,free=281927600,total=521142272
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedFacetPivotWhiteBoxTest, TestConfigOverlay, DeleteLastCustomShardedReplicaTest, TestSimNodeAddedTrigger, TestSystemIdResolver, HdfsThreadLeakTest, CircularListTest, ZkFailoverTest, IndexSizeTriggerMixedBoundsTest, TestPointFields, TestSlowCompositeReaderWrapper, SignatureUpdateProcessorFactoryTest, CSVRequestHandlerTest, CurrencyFieldTypeTest, TestGeoJSONResponseWriter, MergeStrategyTest, TestHashQParserPlugin, LeaderFailoverAfterPartitionTest, CdcrOpsAndBoundariesTest, CoreAdminCreateDiscoverTest, TestFuzzyAnalyzedSuggestions, ConcurrentCreateCollectionTest, SchemaVersionSpecificBehaviorTest, AuditLoggerIntegrationTest, TestQueryTypes, MetricsHistoryIntegrationTest, DocValuesNotIndexedTest, TestStreamBody, SimpleCollectionCreateDeleteTest, JWTAuthPluginIntegrationTest, AddReplicaTest, SearchRateTriggerTest, TestDynamicLoadingUrl, TestSolrJ, TestIndexingPerformance, CloudMLTQParserTest, AutoScalingHandlerTest, TestBulkSchemaConcurrent, TestSolr4Spatial, DistributedFacetExistsSmallTest, TestExpandComponent, MetricsConfigTest, ManagedSchemaRoundRobinCloudTest, TestCustomSort, SpellCheckCollatorTest, ParsingFieldUpdateProcessorsTest, TestScoreJoinQPNoScore, TestHdfsUpdateLog, HttpPartitionTest, TestSolrCloudWithDelegationTokens, TestReversedWildcardFilterFactory, TestDistributedTracing, ConfigSetsAPITest, TestFieldTypeResource, OverseerRolesTest, TestQuerySenderNoQuery, TestDeleteCollectionOnDownNodes, TestManagedResource, AnalyticsQueryTest, TestReRankQParserPlugin, DistributedSpellCheckComponentTest, TestExclusionRuleCollectionAccess, TestAuthenticationFramework, SolrCloudExampleTest, TestChildDocTransformerHierarchy, TestNumericRangeQuery32, TestTermsQParserPlugin, TestJsonFacetsWithNestedObjects, TestStressRecovery, PingRequestHandlerTest, TestSerializedLuceneMatchVersion, ReindexCollectionTest, DirectoryFactoryTest, NodeAddedTriggerIntegrationTest, TestOrdValues, NodeMutatorTest, AtomicUpdatesTest, SolrCoreTest, RestoreTriggerStateTest, UpdateParamsTest, BigEndianAscendingWordDeserializerTest, SignificantTermsQParserPluginTest, OverseerTest, BadCopyFieldTest, TaggingAttributeTest, V2StandaloneTest, FieldMutatingUpdateProcessorTest, HdfsNNFailoverTest, SolrRequestParserTest, TestSolrCoreProperties, TestSolrTestCaseJ4, TestTrie, ProtectedTermFilterFactoryTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, AssignBackwardCompatibilityTest, ChaosMonkeyNothingIsSafeTest, DeleteReplicaTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DeleteReplicaTest -Dtests.seed=1D50F113160491FD -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J3 | DeleteReplicaTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1D50F113160491FD]:0)
   [junit4] Completed [445/910 (1!)] on J3 in 21.92s, 3 tests, 2 errors <<< FAILURES!

[...truncated 46347 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/1173/consoleText

[repro] Revision: 4780100d6272cd3b21f23f5073609e1546fe316f

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC"
[repro] Repro line:  ant test  -Dtestcase=DeleteReplicaTest -Dtests.method=deleteReplicaAndVerifyDirectoryCleanup -Dtests.seed=1D50F113160491FD -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=DeleteReplicaTest -Dtests.seed=1D50F113160491FD -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       DeleteReplicaTest
[repro] ant compile-test

[...truncated 3472 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.DeleteReplicaTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Dtests.seed=1D50F113160491FD -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 121 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.DeleteReplicaTest
[repro] Exiting with code 0

[...truncated 73 lines...]