You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/05/30 15:51:26 UTC

[JENKINS-Experimental-GC] Lucene-Solr-master-MacOSX (64bit/jdk-14.0.1) - Build # 6091 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6091/
Java: 64bit/jdk-14.0.1 -XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestRequestForwarding

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestRequestForwarding:     1) Thread[id=15333, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestRequestForwarding]         at java.base@14.0.1/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestRequestForwarding: 
   1) Thread[id=15333, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestRequestForwarding]
        at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([F317341174D3007A]:0)




Build Log:
[...truncated 14612 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestRequestForwarding
   [junit4]   2> 2017635 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 2017636 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/data-dir-130-001
   [junit4]   2> 2017636 WARN  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 2017636 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2017637 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="None") w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 2017637 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2017642 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testMultiCollectionQuery
   [junit4]   2> 2017643 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001
   [junit4]   2> 2017643 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2017645 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 2017645 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2017754 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer start zk server on port: 55353
   [junit4]   2> 2017754 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:55353
   [junit4]   2> 2017755 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:55353
   [junit4]   2> 2017755 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 55353
   [junit4]   2> 2017758 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017763 INFO  (zkConnectionManagerCallback-10189-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017763 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017767 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017769 INFO  (zkConnectionManagerCallback-10191-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017769 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017770 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017773 INFO  (zkConnectionManagerCallback-10193-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017774 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 2017882 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 2017883 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 2017883 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 2017883 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14.0.1+7
   [junit4]   2> 2017889 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2017890 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2017890 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2017890 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3d9e9817{/solr,null,AVAILABLE}
   [junit4]   2> 2017892 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.AbstractConnector Started ServerConnector@edafb{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:55360}
   [junit4]   2> 2017892 INFO  (jetty-launcher-10194-thread-3) [     ] o.e.j.s.Server Started @2018382ms
   [junit4]   2> 2017892 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55360}
   [junit4]   2> 2017893 ERROR (jetty-launcher-10194-thread-3) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 2017893 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-05-30T15:14:26.075699Z
   [junit4]   2> 2017894 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@470aff7e{/solr,null,AVAILABLE}
   [junit4]   2> 2017895 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017895 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@369947b0{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:55361}
   [junit4]   2> 2017895 INFO  (jetty-launcher-10194-thread-2) [     ] o.e.j.s.Server Started @2018385ms
   [junit4]   2> 2017895 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55361}
   [junit4]   2> 2017895 ERROR (jetty-launcher-10194-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2017895 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-05-30T15:14:26.078109Z
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2017896 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e2713d4{/solr,null,AVAILABLE}
   [junit4]   2> 2017897 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017899 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@757fd0f8{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:55362}
   [junit4]   2> 2017900 INFO  (jetty-launcher-10194-thread-1) [     ] o.e.j.s.Server Started @2018389ms
   [junit4]   2> 2017900 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55362}
   [junit4]   2> 2017900 ERROR (jetty-launcher-10194-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2017900 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2017901 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 9.0.0
   [junit4]   2> 2017901 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2017901 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 2017901 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-05-30T15:14:26.083277Z
   [junit4]   2> 2017904 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2017905 INFO  (zkConnectionManagerCallback-10198-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017906 INFO  (zkConnectionManagerCallback-10196-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017906 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017906 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017907 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2017907 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2017911 INFO  (zkConnectionManagerCallback-10200-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017912 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2017913 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2018036 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 2018038 WARN  (jetty-launcher-10194-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54f1705[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018038 WARN  (jetty-launcher-10194-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54f1705[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018041 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 2018042 WARN  (jetty-launcher-10194-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2dc91f14[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018042 WARN  (jetty-launcher-10194-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2dc91f14[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018043 WARN  (jetty-launcher-10194-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@f689204[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018043 WARN  (jetty-launcher-10194-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@f689204[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018045 INFO  (jetty-launcher-10194-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 2018046 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2018049 WARN  (jetty-launcher-10194-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1b1b2431[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018049 WARN  (jetty-launcher-10194-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1b1b2431[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018050 INFO  (jetty-launcher-10194-thread-3) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 2018053 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2018054 INFO  (zkConnectionManagerCallback-10218-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2018054 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2018057 INFO  (zkConnectionManagerCallback-10222-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2018057 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2018139 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 2018140 WARN  (jetty-launcher-10194-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@476c27de[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018140 WARN  (jetty-launcher-10194-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@476c27de[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018144 WARN  (jetty-launcher-10194-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@77f02a92[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018144 WARN  (jetty-launcher-10194-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@77f02a92[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 2018146 INFO  (jetty-launcher-10194-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55353/solr
   [junit4]   2> 2018149 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2018155 INFO  (zkConnectionManagerCallback-10237-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2018155 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2018225 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55362_solr
   [junit4]   2> 2018228 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.Overseer Overseer (id=72074003526320136-127.0.0.1:55362_solr-n_0000000000) starting
   [junit4]   2> 2018287 INFO  (OverseerStateUpdate-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:55362_solr
   [junit4]   2> 2018290 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55362_solr
   [junit4]   2> 2018304 INFO  (OverseerStateUpdate-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2018326 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 2018328 WARN  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 2018396 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2018514 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55362.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018527 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55362.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018528 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55362.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018534 INFO  (jetty-launcher-10194-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node1/.
   [junit4]   2> 2018630 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2018653 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:55361_solr as DOWN
   [junit4]   2> 2018659 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 2018659 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55361_solr
   [junit4]   2> 2018668 INFO  (zkCallback-10236-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2018669 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2018678 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 2018679 WARN  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 2018724 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2018759 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55361.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018786 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55361.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018786 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55361.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018791 INFO  (jetty-launcher-10194-thread-2) [n:127.0.0.1:55361_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node2/.
   [junit4]   2> 2018816 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2018840 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:55360_solr as DOWN
   [junit4]   2> 2018847 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 2018847 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55360_solr
   [junit4]   2> 2018854 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2018855 INFO  (zkCallback-10236-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2018855 INFO  (zkCallback-10221-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2018862 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 2018863 WARN  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 2018908 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2018947 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55360.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018960 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55360.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018961 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55360.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2018965 INFO  (jetty-launcher-10194-thread-3) [n:127.0.0.1:55360_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node3/.
   [junit4]   2> 2018993 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=3
   [junit4]   2> 2018995 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2019001 INFO  (zkConnectionManagerCallback-10266-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2019001 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2019016 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2019026 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55353/solr ready
   [junit4]   2> 2019031 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2019036 INFO  (zkConnectionManagerCallback-10268-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2019036 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2020682 INFO  (qtp248063314-15375) [n:127.0.0.1:55360_solr     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&maxShardsPerNode=1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2020691 INFO  (OverseerThreadFactory-10244-thread-1-processing-n:127.0.0.1:55362_solr) [n:127.0.0.1:55362_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 2020837 INFO  (qtp248063314-15377) [n:127.0.0.1:55360_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=1
   [junit4]   2> 2020843 INFO  (qtp945729227-15371) [n:127.0.0.1:55361_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 2020849 INFO  (qtp2075664895-15383) [n:127.0.0.1:55362_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 2020850 INFO  (qtp2075664895-15383) [n:127.0.0.1:55362_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 2020855 INFO  (qtp248063314-15374) [n:127.0.0.1:55360_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 2020857 INFO  (qtp945729227-15379) [n:127.0.0.1:55361_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 2020859 INFO  (qtp2075664895-15372) [n:127.0.0.1:55362_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 2020895 INFO  (OverseerStateUpdate-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55362/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55362_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2020916 INFO  (OverseerStateUpdate-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55361/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55361_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2021119 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr    x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2021121 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr    x:collection1_shard2_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2021152 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 2021153 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 2021177 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 2021186 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 2022273 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2022380 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2022480 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from configset conf1, trusted=true
   [junit4]   2> 2022483 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55361.solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2022483 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node2/./collection1_shard2_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node2/./collection1_shard2_replica_n2/data/]
   [junit4]   2> 2022487 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=22, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6295328192259027]
   [junit4]   2> 2022623 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 2022624 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55362.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a8bb696
   [junit4]   2> 2022624 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node1/./collection1_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestRequestForwarding_F317341174D3007A-001/tempDir-001/node1/./collection1_shard1_replica_n1/data/]
   [junit4]   2> 2022638 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=22, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6295328192259027]
   [junit4]   2> 2023221 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2023221 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2023231 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2023231 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2023231 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2023231 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2023233 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=104.0185546875, floorSegmentMB=0.7880859375, forceMergeDeletesPctAllowed=13.541406204523042, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=26.07578649317287
   [junit4]   2> 2023234 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2023234 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2023238 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=104.0185546875, floorSegmentMB=0.7880859375, forceMergeDeletesPctAllowed=13.541406204523042, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=26.07578649317287
   [junit4]   2> 2023238 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2023243 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2023276 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct, classname=DirectSolrSpellChecker, field=lowerfilt, minQueryLength=3}
   [junit4]   2> 2023299 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct, classname=DirectSolrSpellChecker, field=lowerfilt, minQueryLength=3}
   [junit4]   2> 2023425 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 2023425 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1668128882406981632
   [junit4]   2> 2023430 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2023437 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2023437 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2023437 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2023438 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2023443 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2023444 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2023445 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2023445 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 2023445 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2023445 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1668128882427953152
   [junit4]   2> 2023446 INFO  (searcherExecutor-10270-thread-1-processing-n:127.0.0.1:55361_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SolrCore [collection1_shard2_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 2023449 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 2023450 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
   [junit4]   2> 2023451 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2023452 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2023452 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2023452 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2023452 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2023453 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2023454 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2023461 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2023464 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2023467 INFO  (searcherExecutor-10272-thread-1-processing-n:127.0.0.1:55362_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 2023472 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 2023472 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 2023476 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2023476 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2023476 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/
   [junit4]   2> 2023477 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2023478 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.SyncStrategy http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 2023478 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72074003526320134-core_node4-n_0000000000
   [junit4]   2> 2023486 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/ shard2
   [junit4]   2> 2023487 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2023487 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2023487 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/
   [junit4]   2> 2023488 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2023490 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 2023490 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72074003526320136-core_node3-n_0000000000
   [junit4]   2> 2023492 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023493 INFO  (zkCallback-10217-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023496 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2023500 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 2023505 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2386
   [junit4]   2> 2023615 INFO  (zkCallback-10236-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023615 INFO  (zkCallback-10217-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023616 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023616 INFO  (zkCallback-10217-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023620 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2023626 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2507
   [junit4]   2> 2023632 INFO  (qtp248063314-15375) [n:127.0.0.1:55360_solr     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 2023634 INFO  (OverseerCollectionConfigSetProcessor-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2023734 INFO  (zkCallback-10217-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023734 INFO  (zkCallback-10221-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023734 INFO  (zkCallback-10217-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023734 INFO  (zkCallback-10236-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023734 INFO  (zkCallback-10236-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023734 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2023737 INFO  (qtp248063314-15375) [n:127.0.0.1:55360_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&maxShardsPerNode=1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3054
   [junit4]   2> 2023737 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: collection1
   [junit4]   2> 2023758 INFO  (qtp945729227-15379) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671929&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023758 INFO  (qtp2075664895-15382) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671929&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023760 INFO  (qtp2075664895-15383) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={q=cat:football^2} hits=0 status=0 QTime=13
   [junit4]   2> 2023765 INFO  (qtp2075664895-15382) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671945&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023765 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671945&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023767 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={q=cat:football^2} hits=0 status=0 QTime=4
   [junit4]   2> 2023774 INFO  (qtp2075664895-15382) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671951&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023775 INFO  (qtp945729227-15366) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671951&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023776 INFO  (qtp945729227-15371) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={q=cat:football^2} hits=0 status=0 QTime=7
   [junit4]   2> 2023780 INFO  (qtp945729227-15366) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671961&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023781 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671961&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023782 INFO  (qtp945729227-15380) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={q=cat:football^2} hits=0 status=0 QTime=3
   [junit4]   2> 2023788 INFO  (qtp2075664895-15382) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&_forwardedCount=1&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671969&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023789 INFO  (qtp945729227-15371) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&_forwardedCount=1&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671969&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023789 INFO  (qtp2075664895-15372) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={q=cat:football^2&_forwardedCount=1} hits=0 status=0 QTime=2
   [junit4]   2> 2023794 INFO  (qtp945729227-15379) [n:127.0.0.1:55361_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2 ] o.a.s.c.S.Request [collection1_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55361/solr/collection1_shard2_replica_n2/&_forwardedCount=1&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671975&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023794 INFO  (qtp2075664895-15383) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:55362/solr/collection1_shard1_replica_n1/&_forwardedCount=1&rows=10&version=2&q=cat:football^2&omitHeader=false&NOW=1590851671975&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2023795 INFO  (qtp2075664895-15384) [n:127.0.0.1:55362_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/solr path=/select params={q=cat:football^2&_forwardedCount=1} hits=0 status=0 QTime=1
   [junit4]   2> 2023901 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1466168262
   [junit4]   2> 2023901 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55362_solr
   [junit4]   2> 2023901 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=565306331
   [junit4]   2> 2023902 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55361_solr
   [junit4]   2> 2023903 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=26028965
   [junit4]   2> 2023903 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55360_solr
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:55362_solr as DOWN
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:55361_solr as DOWN
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 2023904 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:55360_solr as DOWN
   [junit4]   2> 2023918 INFO  (coreCloseExecutor-10292-thread-1) [n:127.0.0.1:55361_solr     ] o.a.s.c.SolrCore [collection1_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@5c5d121c
   [junit4]   2> 2023918 INFO  (coreCloseExecutor-10292-thread-1) [n:127.0.0.1:55361_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n2 tag=SolrCore@5c5d121c
   [junit4]   2> 2023918 INFO  (coreCloseExecutor-10292-thread-1) [n:127.0.0.1:55361_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25ce8fc6: rootName = solr_55361, domain = solr.core.collection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n2/com.codahale.metrics.MetricRegistry@58dcb91f
   [junit4]   2> 2023919 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 2023919 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3a8f3fb5: rootName = solr_55360, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@62a81c84
   [junit4]   2> 2023920 INFO  (coreCloseExecutor-10293-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6e3aeeda
   [junit4]   2> 2023920 INFO  (coreCloseExecutor-10293-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1 tag=SolrCore@6e3aeeda
   [junit4]   2> 2023920 INFO  (coreCloseExecutor-10293-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ee43b95: rootName = solr_55362, domain = solr.core.collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n1/com.codahale.metrics.MetricRegistry@4d52aaa0
   [junit4]   2> 2024028 INFO  (zkCallback-10217-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2024028 INFO  (zkCallback-10217-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2024028 INFO  (zkCallback-10217-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2024030 INFO  (zkCallback-10236-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2024030 INFO  (zkCallback-10236-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2024149 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 2024149 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d8752f8: rootName = solr_55360, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@4fe016a8
   [junit4]   2> 2024238 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 2024239 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2248e48b: rootName = solr_55360, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@7410b6
   [junit4]   2> 2024239 INFO  (jetty-closer-10281-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 2024269 INFO  (coreCloseExecutor-10293-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader tag=SolrCore@6e3aeeda
   [junit4]   2> 2024270 INFO  (coreCloseExecutor-10293-thread-1) [n:127.0.0.1:55362_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 2024274 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 2024275 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@65b5802: rootName = solr_55362, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@17bc2a38
   [junit4]   2> 2024275 INFO  (coreCloseExecutor-10292-thread-1) [n:127.0.0.1:55361_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader tag=SolrCore@5c5d121c
   [junit4]   2> 2024276 INFO  (coreCloseExecutor-10292-thread-1) [n:127.0.0.1:55361_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 2024279 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 2024279 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fcbe3c5: rootName = solr_55361, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@30798d63
   [junit4]   2> 2024284 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 2024284 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c334ccf: rootName = solr_55362, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@4fe016a8
   [junit4]   2> 2024292 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 2024292 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3507ee6f: rootName = solr_55362, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@7410b6
   [junit4]   2> 2024292 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 2024294 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 2024294 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60bd112c: rootName = solr_55361, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@4fe016a8
   [junit4]   2> 2024294 INFO  (closeThreadPool-10297-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72074003526320136-127.0.0.1:55362_solr-n_0000000000) closing
   [junit4]   2> 2024295 INFO  (OverseerStateUpdate-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [n:127.0.0.1:55362_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55362_solr
   [junit4]   2> 2024296 INFO  (OverseerAutoScalingTriggerThread-72074003526320136-127.0.0.1:55362_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2024296 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 2024296 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@404dd275: rootName = solr_55361, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@7410b6
   [junit4]   2> 2024296 INFO  (jetty-closer-10281-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 2024300 INFO  (closeThreadPool-10297-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72074003526320136-127.0.0.1:55362_solr-n_0000000000) closing
   [junit4]   2> 2024352 INFO  (zkCallback-10217-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 2024352 INFO  (zkCallback-10236-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 2024359 INFO  (jetty-closer-10281-thread-3) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@edafb{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 2024361 INFO  (jetty-closer-10281-thread-3) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3d9e9817{/solr,null,UNAVAILABLE}
   [junit4]   2> 2024361 INFO  (jetty-closer-10281-thread-3) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 2024460 INFO  (jetty-closer-10281-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72074003526320136-127.0.0.1:55362_solr-n_0000000000) closing
   [junit4]   2> 2024465 INFO  (jetty-closer-10281-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@369947b0{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 2024465 INFO  (jetty-closer-10281-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@470aff7e{/solr,null,UNAVAILABLE}
   [junit4]   2> 2024465 INFO  (jetty-closer-10281-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 2024468 INFO  (jetty-closer-10281-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@757fd0f8{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 2024469 INFO  (jetty-closer-10281-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3e2713d4{/solr,null,UNAVAILABLE}
   [junit4]   2> 2024469 INFO  (jetty-closer-10281-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 2024472 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 2024693 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 	4	/solr/clusterprops.json
   [junit4]   2> 	3	/solr/packages.json
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 	2	/solr/collections/collection1/terms/shard2
   [junit4]   2> 	2	/solr/configs/conf1
   [junit4]   2> 	2	/solr/collections/collection1/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	27	/solr/collections/collection1/state.json
   [junit4]   2> 	4	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	21	/solr/overseer/queue
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	4	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 2024703 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:55353
   [junit4]   2> 2024703 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:55353
   [junit4]   2> 2024703 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 55353
   [junit4]   2> 2024704 INFO  (TEST-TestRequestForwarding.testMultiCollectionQuery-seed#[F317341174D3007A]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testMultiCollectionQuery
   [junit4]   2> 2024706 INFO  (SUITE-TestRequestForwarding-seed#[F317341174D3007A]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> May 30, 2020 3:14:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> May 30, 2020 3:14:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestRequestForwarding: 
   [junit4]   2>    1) Thread[id=15333, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestRequestForwarding]
   [junit4]   2>         at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]   2> May 30, 2020 3:14:43 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=15333, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestRequestForwarding]
   [junit4]   2> 2034818 ERROR (SessionTracker) [     ] o.a.z.s.ZooKeeperCriticalThread Severe unrecoverable error, from thread : SessionTracker
   [junit4]   2>           => java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.base/java.lang.Thread.sleep(Native Method)
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method) [?:?]
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) ~[zookeeper-3.5.7.jar:3.5.7]
   [junit4]   2> May 30, 2020 3:14:43 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {}, docValues:{}, maxPointsInLeafNode=753, maxMBSortInHeap=5.74245148623403, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3542d409), locale=ar-DZ, timezone=Etc/GMT+0
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 14.0.1 (64-bit)/cpus=6,threads=1,free=260046848,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SolrJmxReporterTest, ScheduledTriggerIntegrationTest, CustomCollectionTest, HdfsDirectoryTest, SolrIndexSearcherTest, TestStressLucene, BasicAuthIntegrationTest, TestPivotHelperCode, TestRestoreCore, TestBinaryResponseWriter, ConjunctionSolrSpellCheckerTest, PreAnalyzedUpdateProcessorTest, TestInPlaceUpdateWithRouteField, TestAnalyzedSuggestions, SyncSliceTest, TestManagedSchema, TestFieldCollectionResource, TestExactSharedStatsCache, TestMultiWordSynonyms, ClusterStateMockUtilTest, QueryEqualityTest, SolrTestCaseJ4Test, DateRangeFieldTest, GraphQueryTest, LeaderVoteWaitTimeoutTest, BooleanFieldTest, ConnectionReuseTest, ExternalFileFieldSortTest, TestSubQueryTransformer, TestNumericTokenStream, TestCloudRecovery, TestSolrCoreSnapshots, DistributedFacetPivotLongTailTest, TestHighFrequencyDictionaryFactory, TermVectorComponentTest, PeerSyncWithLeaderTest, DeleteShardTest, XsltUpdateRequestHandlerTest, RandomizedTaggerTest, TestJsonFacetsStatsParsing, SSLMigrationTest, EchoParamsTest, TestDeprecatedFilters, TestSimPolicyCloud, SuggesterFSTTest, TestLuceneIndexBackCompat, JWTAuthPluginTest, TestHdfsBackupRestoreCore, TestShardHandlerFactory, DistributedQueueTest, TestSolrFieldCacheBean, CollectionTooManyReplicasTest, FieldAnalysisRequestHandlerTest, TestDistributedSearch, TestFaceting, TestLocalStatsCacheCloud, CurrencyRangeFacetCloudTest, SolrIndexMetricsTest, MultiThreadedOCPTest, TestApiFramework, XCJFQueryTest, CategoryRoutedAliasUpdateProcessorTest, TestShortCircuitedRequests, TestSmileRequest, TestWriterPerf, ChaosMonkeySafeLeaderTest, NumericFieldsTest, TestCollapseQParserPlugin, CachingDirectoryFactoryTest, IndexSizeEstimatorTest, RecoveryAfterSoftCommitTest, ReplaceNodeTest, CreateRoutedAliasTest, SliceStateTest, BigEndianAscendingWordDeserializerTest, TestDelegationWithHadoopAuth, TestRandomRequestDistribution, TestReload, MaxSizeAutoCommitTest, TestFiltering, OverseerStatusTest, TestPolicyCloud, TestJavabinTupleStreamParser, PrimitiveFieldTypeTest, TestManagedStopFilterFactory, ShowFileRequestHandlerTest, SpellCheckCollatorWithCollapseTest, TestRequestStatusCollectionAPI, TestPseudoReturnFields, TestConfigSetProperties, CdcrRequestHandlerTest, DistribDocExpirationUpdateProcessorTest, LeaderFailoverAfterPartitionTest, MissingSegmentRecoveryTest, MoveReplicaHDFSFailoverTest, MoveReplicaHDFSTest, MoveReplicaTest, TestCloudInspectUtil, TestCloudPhrasesIdentificationComponent, TestCloudSearcherWarming, TestExactStatsCacheCloud, TestLeaderElectionWithEmptyReplica, TestQueryingOnDownCollection, TestRSAKeyPair, TestRequestForwarding]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRequestForwarding -Dtests.seed=F317341174D3007A -Dtests.slow=true -Dtests.locale=ar-DZ -Dtests.timezone=Etc/GMT+0 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestRequestForwarding (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestRequestForwarding: 
   [junit4]    >    1) Thread[id=15333, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestRequestForwarding]
   [junit4]    >         at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
   [junit4]    >         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F317341174D3007A]:0)
   [junit4] Completed [539/913 (1!)] on J1 in 17.45s, 1 test, 1 error <<< FAILURES!

[...truncated 46169 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6091/consoleText

[repro] Revision: a6a02ac0e5116f4cf061fd3afeb930c3cc379f5a

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC"
[repro] Repro line:  ant test  -Dtestcase=TestRequestForwarding -Dtests.seed=F317341174D3007A -Dtests.slow=true -Dtests.locale=ar-DZ -Dtests.timezone=Etc/GMT+0 -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestRequestForwarding
[repro] ant compile-test

[...truncated 3147 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestRequestForwarding" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC" -Dtests.seed=F317341174D3007A -Dtests.slow=true -Dtests.locale=ar-DZ -Dtests.timezone=Etc/GMT+0 -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 81 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.cloud.TestRequestForwarding
[repro] Exiting with code 0

[...truncated 73 lines...]

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-13.0.2) - Build # 6092 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6092/
Java: 64bit/jdk-13.0.2 -XX:-UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaAPI:     1) Thread[id=32363, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaAPI]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaAPI: 
   1) Thread[id=32363, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaAPI]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([3E461CD8626019B7]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaThreadSafety

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaThreadSafety:     1) Thread[id=25067, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaThreadSafety]         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaThreadSafety: 
   1) Thread[id=25067, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaThreadSafety]
        at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
	at __randomizedtesting.SeedInfo.seed([3E461CD8626019B7]:0)




Build Log:
[...truncated 15474 lines...]
   [junit4] Suite: org.apache.solr.schema.TestManagedSchemaThreadSafety
   [junit4]   2> 1660096 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1660096 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J2/temp/solr.schema.TestManagedSchemaThreadSafety_3E461CD8626019B7-001/data-dir-266-001
   [junit4]   2> 1660097 WARN  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
   [junit4]   2> 1660097 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1660098 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1660098 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1660098 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1660099 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1660100 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1660199 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer start zk server on port: 50522
   [junit4]   2> 1660199 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:50522
   [junit4]   2> 1660199 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:50522
   [junit4]   2> 1660199 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 50522
   [junit4]   2> 1660202 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1660217 INFO  (zkConnectionManagerCallback-16561-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1660217 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1660221 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1660226 INFO  (zkConnectionManagerCallback-16563-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1660226 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1660235 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testThreadSafety
   [junit4]   2> 1660236 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1660238 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1660243 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]-EventThread) [     ] o.a.s.c.c.SolrZkClient Submitting job to respond to event WatchedEvent state:SyncConnected type:None path:null
   [junit4]   2> 1660243 INFO  (zkConnectionManagerCallback-16565-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1660243 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1660244 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1660244 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient makePath: /configs/managed-config/schema.xml
   [junit4]   2> 1660247 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient makePath: /configs/managed-config/solrconfig.xml
   [junit4]   2> 1660356 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]-EventThread) [     ] o.a.s.c.c.SolrZkClient Submitting job to respond to event WatchedEvent state:Closed type:None path:null
   [junit4]   2> 1660357 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1660362 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1660365 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]-EventThread) [     ] o.a.s.c.c.SolrZkClient Submitting job to respond to event WatchedEvent state:SyncConnected type:None path:null
   [junit4]   2> 1660365 INFO  (zkConnectionManagerCallback-16568-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1660365 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1660365 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1660468 DEBUG (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]-EventThread) [     ] o.a.s.c.c.SolrZkClient Submitting job to respond to event WatchedEvent state:Closed type:None path:null
   [junit4]   2> 1660469 INFO  (TEST-TestManagedSchemaThreadSafety.testThreadSafety-seed#[3E461CD8626019B7]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testThreadSafety
   [junit4] IGNOR/A 0.24s J2 | TestManagedSchemaThreadSafety.testThreadSafety
   [junit4]    > Assumption #1: SOLR-11606: ByteBuddy used by Mockito is not working with this JVM version.
   [junit4]   2> 1660470 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 1660686 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:50522
   [junit4]   2> 1660686 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:50522
   [junit4]   2> 1660686 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 50522
   [junit4]   2> 1660687 INFO  (SUITE-TestManagedSchemaThreadSafety-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> May 30, 2020 7:41:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> May 30, 2020 7:41:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaThreadSafety: 
   [junit4]   2>    1) Thread[id=25067, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaThreadSafety]
   [junit4]   2>         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]   2> May 30, 2020 7:41:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=25067, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaThreadSafety]
   [junit4]   2> 1670792 ERROR (SessionTracker) [     ] o.a.z.s.ZooKeeperCriticalThread Severe unrecoverable error, from thread : SessionTracker
   [junit4]   2>           => java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.base/java.lang.Thread.sleep(Native Method)
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method) [?:?]
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) ~[zookeeper-3.5.7.jar:3.5.7]
   [junit4]   2> May 30, 2020 7:41:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7e942031), locale=ga-IE, timezone=Europe/Chisinau
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 13.0.2 (64-bit)/cpus=6,threads=1,free=132398584,total=456790016
   [junit4]   2> NOTE: All tests run in this JVM: [TestReRankQParserPlugin, ReplicationFactorTest, HLLUtilTest, TestSizeLimitedDistributedMap, CdcrUpdateLogTest, TestBulkSchemaAPI, TestStressRecovery, TestSimComputePlanAction, DistributedQueryComponentOptimizationTest, HttpSolrCallGetCoreTest, InfoHandlerTest, TestManagedSynonymFilterFactory, DeleteStatusTest, TestComplexPhraseLeadingWildcard, MetricTriggerIntegrationTest, UpdateRequestProcessorFactoryTest, TestStressCloudBlindAtomicUpdates, HdfsAutoAddReplicasIntegrationTest, ConvertedLegacyTest, TestDirectoryFactory, OverseerModifyCollectionTest, TestOrdValues, TestImpersonationWithHadoopAuth, DeleteInactiveReplicaTest, HdfsChaosMonkeyNothingIsSafeTest, DocValuesMissingTest, TestSQLHandler, TestSweetSpotSimilarityFactory, TestFoldingMultitermQuery, ClassificationUpdateProcessorFactoryTest, CollectionsAPIAsyncDistributedZkTest, TestSolrQueryResponse, TestCSVResponseWriter, HdfsLockFactoryTest, RootFieldTest, TestSimGenericDistributedQueue, TestExactStatsCache, MinimalSchemaTest, TestJsonFacetsWithNestedObjects, TestSystemCollAutoCreate, TestReloadDeadlock, TestCloudNestedDocsSort, TestConfigSets, ConfigureRecoveryStrategyTest, TestZkChroot, TriggerCooldownIntegrationTest, HdfsWriteToMultipleCollectionsTest, OverseerTaskQueueTest, ConcurrentCreateRoutedAliasTest, TestPhraseSuggestions, TestInfoStreamLogging, TestLegacyTerms, SolrMetricReporterTest, DistributedDebugComponentTest, TestTrackingShardHandlerFactory, HighlighterConfigTest, TestTestInjection, TestMacroExpander, OpenExchangeRatesOrgProviderTest, TestAuthorizationFramework, TestCodecSupport, TestLogWatcher, HttpPartitionOnCommitTest, StatelessScriptUpdateProcessorFactoryTest, TestFieldResource, TestReplicationHandler, TestHttpServletCarrier, ChaosMonkeySafeLeaderWithPullReplicasTest, TestCollectionAPIs, MoveReplicaHDFSFailoverTest, AddSchemaFieldsUpdateProcessorFactoryTest, ZkSolrClientTest, TestDownShardTolerantSearch, QueryParsingTest, SuggesterTSTTest, SystemCollectionCompatTest, SortSpecParsingTest, BasicAuthOnSingleNodeTest, TestCollectionsAPIViaSolrCloudCluster, CursorMarkTest, BigEndianAscendingWordSerializerTest, TestTermsQParserPlugin, SolrPluginUtilsTest, HdfsCollectionsAPIDistributedZkTest, BasicDistributedZkTest, TestSimScenario, SOLR749Test, RegexBoostProcessorTest, SpatialRPTFieldTypeTest, TestSolr4Spatial, TestRandomFaceting, TestWordDelimiterFilterFactory, TestEmbeddedSolrServerAdminHandler, TestJettySolrRunner, ConnectionReuseTest, ActionThrottleTest, AddReplicaTest, BasicZkTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, ClusterStateMockUtilTest, ClusterStateTest, ClusterStateUpdateTest, ConfigSetsAPITest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderTest, LeaderElectionContextKeyTest, LeaderElectionIntegrationTest, LeaderFailoverAfterPartitionTest, MoveReplicaHDFSTest, MoveReplicaTest, RecoveryAfterSoftCommitTest, SSLMigrationTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest, SliceStateTest, SolrCLIZkUtilsTest, TestCloudInspectUtil, TestCloudPseudoReturnFields, TestCloudRecovery2, TestConfigSetsAPI, TestDynamicFieldNamesIndexCorrectly, TestLeaderElectionZkExpiry, TestRebalanceLeaders, TestUtilizeNode, VMParamsZkACLAndCredentialsProvidersTest, ZkCLITest, TestReplicaProperties, AutoScalingHandlerTest, NodeAddedTriggerTest, NodeMarkersRegistrationTest, RestoreTriggerStateTest, TestSimNodeAddedTrigger, TestSnapshotCloudManager, CdcrOpsAndBoundariesTest, CdcrReplicationHandlerTest, CdcrRequestHandlerTest, CdcrVersionReplicationTest, CdcrWithNodesRestartsTest, HDFSCollectionsAPITest, ZkDynamicConfigTest, AlternateDirectoryTest, CoreSorterTest, DirectoryFactoryTest, ExitableDirectoryReaderTest, HdfsDirectoryFactoryTest, TestNRTOpen, TestQuerySenderListener, TestReloadAndDeleteDocs, TestSolrXml, TestSolrCloudSnapshots, TestSQLHandlerNonCloud, V2ApiIntegrationTest, XsltUpdateRequestHandlerTest, CoreAdminHandlerTest, MetricsHandlerTest, ZookeeperReadAPITest, CloudReplicaSourceTest, DistributedExpandComponentTest, DistributedFacetPivotLargeTest, DistributedSpellCheckComponentTest, StatsComponentTest, TestNumericRangeQuery32, SolrSlf4jReporterTest, TestRemoteStreaming, TestStreamBody, TestJavabinTupleStreamParser, TestPHPSerializedResponseWriter, TestChildDocTransformer, TestExplainDocTransformer, TestSubQueryTransformerCrossCore, TestManagedResource, TestCopyFieldCollectionResource, TestDynamicFieldCollectionResource, TestFieldTypeCollectionResource, TestFieldTypeResource, TestSchemaNameResource, TestSchemaResource, TestSchemaSimilarityResource, BadIndexSchemaTest, BooleanFieldTest, ChangedSchemaMergeTest, ExternalFileFieldSortTest, IndexSchemaRuntimeFieldTest, PreAnalyzedFieldManagedSchemaCloudTest, TestHalfAndHalfDocValues, TestManagedSchema, TestManagedSchemaThreadSafety]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestManagedSchemaThreadSafety -Dtests.seed=3E461CD8626019B7 -Dtests.slow=true -Dtests.locale=ga-IE -Dtests.timezone=Europe/Chisinau -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J2 | TestManagedSchemaThreadSafety (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaThreadSafety: 
   [junit4]    >    1) Thread[id=25067, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaThreadSafety]
   [junit4]    >         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]    >         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3E461CD8626019B7]:0)
   [junit4] Completed [763/913 (1!)] on J2 in 10.97s, 1 test, 1 error, 1 skipped <<< FAILURES!

[...truncated 16 lines...]
   [junit4] Suite: org.apache.solr.schema.TestManagedSchemaAPI
   [junit4]   2> 1656811 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1656811 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/data-dir-194-001
   [junit4]   2> 1656811 WARN  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 1656812 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1656812 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1656812 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1656813 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001
   [junit4]   2> 1656813 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1656814 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1656814 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1656923 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer start zk server on port: 50485
   [junit4]   2> 1656923 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:50485
   [junit4]   2> 1656923 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:50485
   [junit4]   2> 1656923 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 50485
   [junit4]   2> 1656925 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1656932 INFO  (zkConnectionManagerCallback-22070-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1656932 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1656944 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1656952 INFO  (zkConnectionManagerCallback-22072-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1656953 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1656954 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1656959 INFO  (zkConnectionManagerCallback-22074-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1656959 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657071 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1657071 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1657071 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 13.0.2+8
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1657073 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 13.0.2+8
   [junit4]   2> 1657074 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@61fc1485{/solr,null,AVAILABLE}
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@59f182fd{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:50492}
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.e.j.s.Server Started @1657156ms
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50492}
   [junit4]   2> 1657075 ERROR (jetty-launcher-22075-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1657075 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-05-30T19:41:31.445555Z
   [junit4]   2> 1657076 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657076 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1657076 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1657076 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1657078 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b2558fb{/solr,null,AVAILABLE}
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@712e1e68{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:50494}
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.e.j.s.Server Started @1657160ms
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50494}
   [junit4]   2> 1657079 ERROR (jetty-launcher-22075-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 9.0.0
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2020-05-30T19:41:31.449412Z
   [junit4]   2> 1657079 INFO  (zkConnectionManagerCallback-22077-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1657079 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657081 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657081 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1657081 INFO  (zkConnectionManagerCallback-22079-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1657087 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657093 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1657201 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1657202 WARN  (jetty-launcher-22075-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6b7393dc[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657202 WARN  (jetty-launcher-22075-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6b7393dc[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657210 WARN  (jetty-launcher-22075-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7cfea8b2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657210 WARN  (jetty-launcher-22075-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7cfea8b2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657211 INFO  (jetty-launcher-22075-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50485/solr
   [junit4]   2> 1657213 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657220 INFO  (zkConnectionManagerCallback-22090-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1657220 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657303 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1657306 WARN  (jetty-launcher-22075-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@283608b3[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657306 WARN  (jetty-launcher-22075-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@283608b3[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657313 WARN  (jetty-launcher-22075-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3b481b6a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657313 WARN  (jetty-launcher-22075-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3b481b6a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1657314 INFO  (jetty-launcher-22075-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50485/solr
   [junit4]   2> 1657319 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657333 INFO  (zkConnectionManagerCallback-22101-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1657334 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1658009 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50492_solr
   [junit4]   2> 1658012 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.Overseer Overseer (id=72075053763723269-127.0.0.1:50492_solr-n_0000000000) starting
   [junit4]   2> 1658056 INFO  (OverseerStateUpdate-72075053763723269-127.0.0.1:50492_solr-n_0000000000) [n:127.0.0.1:50492_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:50492_solr
   [junit4]   2> 1658062 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50492_solr
   [junit4]   2> 1658076 INFO  (OverseerStateUpdate-72075053763723269-127.0.0.1:50492_solr-n_0000000000) [n:127.0.0.1:50492_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1658098 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1658099 WARN  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1658123 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1658164 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658175 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658175 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658175 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1658184 INFO  (jetty-launcher-22075-thread-1) [n:127.0.0.1:50492_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/.
   [junit4]   2> 1658211 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:50494_solr as DOWN
   [junit4]   2> 1658221 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1658221 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50494_solr
   [junit4]   2> 1658233 INFO  (zkCallback-22089-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1658234 INFO  (zkCallback-22100-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1658241 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1658242 WARN  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1658269 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1658302 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658309 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658309 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658311 INFO  (jetty-launcher-22075-thread-2) [n:127.0.0.1:50494_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/.
   [junit4]   2> 1658318 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
   [junit4]   2> 1658319 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1658323 INFO  (zkConnectionManagerCallback-22126-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1658323 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1658329 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1658333 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50485/solr ready
   [junit4]   2> 1658372 INFO  (TEST-TestManagedSchemaAPI.test-seed#[3E461CD8626019B7]) [     ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 1658378 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=testschemaapi&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1658392 INFO  (OverseerThreadFactory-22110-thread-1-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection testschemaapi
   [junit4]   2> 1658526 INFO  (qtp1463187051-32396) [n:127.0.0.1:50492_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
   [junit4]   2> 1658526 INFO  (qtp1463187051-32396) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1658529 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1658533 INFO  (qtp1463187051-32393) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
   [junit4]   2> 1658539 INFO  (qtp2146037733-32404) [n:127.0.0.1:50494_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=4
   [junit4]   2> 1658615 INFO  (OverseerStateUpdate-72075053763723269-127.0.0.1:50492_solr-n_0000000000) [n:127.0.0.1:50492_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testschemaapi",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testschemaapi_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:50494/solr",
   [junit4]   2>   "node_name":"127.0.0.1:50494_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1658670 INFO  (OverseerStateUpdate-72075053763723269-127.0.0.1:50492_solr-n_0000000000) [n:127.0.0.1:50492_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testschemaapi",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testschemaapi_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:50492/solr",
   [junit4]   2>   "node_name":"127.0.0.1:50492_solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1658899 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr    x:testschemaapi_shard1_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=testschemaapi_shard1_replica_n2&action=CREATE&numShards=1&collection=testschemaapi&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1658903 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr    x:testschemaapi_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=testschemaapi_shard1_replica_n1&action=CREATE&numShards=1&collection=testschemaapi&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1658922 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1658927 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1658935 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1658963 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1658963 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'testschemaapi_shard1_replica_n2' using configuration from configset conf1, trusted=true
   [junit4]   2> 1658963 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.core.testschemaapi.shard1.replica_n2' (registry 'solr.core.testschemaapi.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658963 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1658964 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2/data/]
   [junit4]   2> 1658967 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1658967 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'testschemaapi_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1658967 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.core.testschemaapi.shard1.replica_n1' (registry 'solr.core.testschemaapi.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1658968 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1/data/]
   [junit4]   2> 1659041 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1659041 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1659043 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1659043 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1659047 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1659048 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1659048 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1659050 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1659050 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1659050 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1668145682495569920
   [junit4]   2> 1659054 INFO  (searcherExecutor-22128-thread-1-processing-n:127.0.0.1:50492_solr x:testschemaapi_shard1_replica_n2 c:testschemaapi s:shard1 r:core_node4) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1659116 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1659116 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1659117 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testschemaapi/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1659117 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testschemaapi/leaders/shard1
   [junit4]   2> 1659125 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1659125 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1659141 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1659145 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1659151 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1659157 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1659157 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1659158 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1668145682608816128
   [junit4]   2> 1659159 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoute in=9996ms
   [junit4]   2> 1659164 INFO  (searcherExecutor-22130-thread-1-processing-n:127.0.0.1:50494_solr x:testschemaapi_shard1_replica_n1 c:testschemaapi s:shard1 r:core_node3) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1659193 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testschemaapi/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 1659193 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testschemaapi/leaders/shard1
   [junit4]   2> 1659666 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1659667 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1659667 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50492/solr/testschemaapi_shard1_replica_n2/
   [junit4]   2> 1659668 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.PeerSync PeerSync: core=testschemaapi_shard1_replica_n2 url=http://127.0.0.1:50492/solr  START replicas=[http://127.0.0.1:50494/solr/testschemaapi_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1659668 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.PeerSync PeerSync: core=testschemaapi_shard1_replica_n2 url=http://127.0.0.1:50492/solr  DONE. We have no versions. sync failed.
   [junit4]   2> 1659671 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.S.Request [testschemaapi_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1659671 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1659671 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1659671 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testschemaapi/leaders/shard1/leader after winning as /collections/testschemaapi/leader_elect/shard1/election/72075053763723269-core_node4-n_0000000000
   [junit4]   2> 1659677 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50492/solr/testschemaapi_shard1_replica_n2/ shard1
   [junit4]   2> 1659783 INFO  (zkCallback-22089-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1659787 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1659792 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=testschemaapi_shard1_replica_n2&action=CREATE&numShards=1&collection=testschemaapi&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=893
   [junit4]   2> 1659893 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1659894 INFO  (zkCallback-22089-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1673499 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=testschemaapi_shard1_replica_n1&action=CREATE&numShards=1&collection=testschemaapi&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=14596
   [junit4]   2> 1673503 INFO  (zkCallback-22100-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1673503 INFO  (zkCallback-22089-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1673504 INFO  (zkCallback-22089-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1673505 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testschemaapi/state.json] for collection [testschemaapi] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1673508 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1673509 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=testschemaapi&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=15130
   [junit4]   2> 1673511 INFO  (OverseerCollectionConfigSetProcessor-72075053763723269-127.0.0.1:50492_solr-n_0000000000) [n:127.0.0.1:50492_solr     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1673519 INFO  (qtp1463187051-32396) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testschemaapi/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
   [junit4]   2> 1673527 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:50492/solr/testschemaapi_shard1_replica_n2/&wt=javabin&version=2}{add=[3 (1668145697664270336)]} 0 2
   [junit4]   2> 1673527 INFO  (qtp1463187051-32396) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3 (1668145697664270336)]} 0 13
   [junit4]   2> 1673536 INFO  (searcherExecutor-22130-thread-1-processing-n:127.0.0.1:50494_solr x:testschemaapi_shard1_replica_n1 c:testschemaapi s:shard1 r:core_node3) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673536 INFO  (searcherExecutor-22128-thread-1-processing-n:127.0.0.1:50492_solr x:testschemaapi_shard1_replica_n2 c:testschemaapi s:shard1 r:core_node4) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673537 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:50492/solr/testschemaapi_shard1_replica_n2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 7
   [junit4]   2> 1673537 INFO  (qtp1463187051-32393) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [testschemaapi_shard1_replica_n2]  webapp=/solr path=/update params={_stateVer_=testschemaapi:6&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 9
   [junit4]   2> 1673539 INFO  (qtp2146037733-32402) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.S.Request [testschemaapi_shard1_replica_n1]  webapp=/solr path=/schema/fields/id params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1673546 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.SchemaManager managed schema loaded . version : 0 
   [junit4]   2> 1673547 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673550 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673551 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.ZkController Persisted config data to node /configs/conf1/managed-schema 
   [junit4]   2> 1673551 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1673552 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1673552 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from Zookeeper
   [junit4]   2> 1673554 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673555 INFO  (Thread-4171) [     ] o.a.s.c.SolrCore config update listener called for core testschemaapi_shard1_replica_n1
   [junit4]   2> 1673556 INFO  (Thread-4172) [     ] o.a.s.c.SolrCore config update listener called for core testschemaapi_shard1_replica_n2
   [junit4]   2> 1673557 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673557 INFO  (zkCallback-22089-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 4 ms
   [junit4]   2> 1673561 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore core reload testschemaapi_shard1_replica_n1
   [junit4]   2> 1673562 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore core reload testschemaapi_shard1_replica_n2
   [junit4]   2> 1673568 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1673571 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1673593 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673596 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673596 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.CoreContainer Reloading SolrCore 'testschemaapi_shard1_replica_n1' using configuration from configset conf1
   [junit4]   2> 1673603 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673605 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673605 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.c.CoreContainer Reloading SolrCore 'testschemaapi_shard1_replica_n2' using configuration from configset conf1
   [junit4]   2> 1673651 INFO  (Thread-4172) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.core.testschemaapi.shard1.replica_n2' (registry 'solr.core.testschemaapi.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1673651 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2/data/]
   [junit4]   2> 1673659 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.core.testschemaapi.shard1.replica_n1' (registry 'solr.core.testschemaapi.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1673659 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1/data/]
   [junit4]   2> 1673723 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1673723 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1673728 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1673729 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1673729 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1673733 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1673733 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1673736 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1673737 INFO  (searcherExecutor-22142-thread-1-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673738 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@67341e15
   [junit4]   2> 1673738 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testschemaapi.shard1.replica_n2 tag=SolrCore@67341e15
   [junit4]   2> 1673738 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7d34976b: rootName = solr_50492, domain = solr.core.testschemaapi.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.testschemaapi.shard1.replica_n2/com.codahale.metrics.MetricRegistry@61b1cc75
   [junit4]   2> 1673748 INFO  (searcherExecutor-22142-thread-1-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673754 INFO  (Thread-4172) [n:127.0.0.1:50492_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testschemaapi.shard1.leader tag=SolrCore@67341e15
   [junit4]   2> 1673769 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1673769 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1673774 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1673775 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1673775 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1673777 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1673777 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1673780 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1673784 INFO  (searcherExecutor-22140-thread-1-processing-n:127.0.0.1:50494_solr x:testschemaapi_shard1_replica_n1 c:testschemaapi s:shard1 r:core_node3) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673786 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.s.ManagedIndexSchema Waiting up to 599 secs for 1 replicas to apply schema update version 1 for collection testschemaapi
   [junit4]   2> 1673787 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from Zookeeper
   [junit4]   2> 1673787 INFO  (searcherExecutor-22140-thread-1-processing-n:127.0.0.1:50494_solr x:testschemaapi_shard1_replica_n1 c:testschemaapi s:shard1 r:core_node3) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673789 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673791 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.c.S.Request [testschemaapi_shard1_replica_n2]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1673791 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.s.ManagedIndexSchema Took 5.0ms for 1 replicas to apply schema update version 1 for collection testschemaapi
   [junit4]   2> 1673791 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.c.S.Request [testschemaapi_shard1_replica_n1]  webapp=/solr path=/schema params={wt=javabin&version=2} status=0 QTime=248
   [junit4]   2> 1673791 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7f88e84f
   [junit4]   2> 1673792 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testschemaapi.shard1.replica_n1 tag=SolrCore@7f88e84f
   [junit4]   2> 1673792 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@76330347: rootName = solr_50494, domain = solr.core.testschemaapi.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testschemaapi.shard1.replica_n1/com.codahale.metrics.MetricRegistry@31575b12
   [junit4]   2> 1673793 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673793 INFO  (zkCallback-22100-thread-2) [     ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 5 ms
   [junit4]   2> 1673801 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1673801 INFO  (qtp2146037733-32403) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testschemaapi.shard1.leader tag=SolrCore@7f88e84f
   [junit4]   2> 1673814 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr     ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :reload with params name=testschemaapi&action=RELOAD&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1673821 INFO  (OverseerThreadFactory-22110-thread-2-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=RELOAD, asyncId=null
   [junit4]   2> 1673821 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673824 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673824 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.c.CoreContainer Reloading SolrCore 'testschemaapi_shard1_replica_n1' using configuration from configset conf1
   [junit4]   2> 1673836 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr    x:testschemaapi_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1673851 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr    x:testschemaapi_shard1_replica_n2 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1673865 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr    x:testschemaapi_shard1_replica_n2 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1673865 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr    x:testschemaapi_shard1_replica_n2 ] o.a.s.c.CoreContainer Reloading SolrCore 'testschemaapi_shard1_replica_n2' using configuration from configset conf1
   [junit4]   2> 1673929 INFO  (Thread-4171) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.core.testschemaapi.shard1.replica_n1' (registry 'solr.core.testschemaapi.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1673929 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1/data/]
   [junit4]   2> 1673948 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50492.solr.core.testschemaapi.shard1.replica_n2' (registry 'solr.core.testschemaapi.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1673948 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node1/./testschemaapi_shard1_replica_n2/data/]
   [junit4]   2> 1674032 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1674032 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1674036 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1674037 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1674038 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1674039 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1674039 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1674041 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1674062 INFO  (searcherExecutor-22152-thread-1-processing-n:127.0.0.1:50494_solr) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674064 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1ccc3a12
   [junit4]   2> 1674064 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testschemaapi.shard1.replica_n1 tag=SolrCore@1ccc3a12
   [junit4]   2> 1674064 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12569f90: rootName = solr_50494, domain = solr.core.testschemaapi.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testschemaapi.shard1.replica_n1/com.codahale.metrics.MetricRegistry@31575b12
   [junit4]   2> 1674065 INFO  (searcherExecutor-22152-thread-1-processing-n:127.0.0.1:50494_solr) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674071 INFO  (Thread-4171) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testschemaapi.shard1.leader tag=SolrCore@1ccc3a12
   [junit4]   2> 1674076 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1674077 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1674086 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr    x:testschemaapi_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
   [junit4]   2> 1674100 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr    x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 1674103 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr    x:testschemaapi_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1674103 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr    x:testschemaapi_shard1_replica_n1 ] o.a.s.c.CoreContainer Reloading SolrCore 'testschemaapi_shard1_replica_n1' using configuration from configset conf1
   [junit4]   2> 1674106 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1674107 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1674108 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1674109 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1674110 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1674111 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1674113 INFO  (searcherExecutor-22154-thread-1-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674118 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@2489a62d
   [junit4]   2> 1674128 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testschemaapi.shard1.replica_n2 tag=SolrCore@2489a62d
   [junit4]   2> 1674128 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d4a6b59: rootName = solr_50492, domain = solr.core.testschemaapi.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.testschemaapi.shard1.replica_n2/com.codahale.metrics.MetricRegistry@61b1cc75
   [junit4]   2> 1674129 INFO  (searcherExecutor-22154-thread-1-processing-n:127.0.0.1:50492_solr) [n:127.0.0.1:50492_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674167 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testschemaapi.shard1.leader tag=SolrCore@2489a62d
   [junit4]   2> 1674169 INFO  (qtp1463187051-32394) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=testschemaapi_shard1_replica_n2&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=345
   [junit4]   2> 1674184 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr c:testschemaapi s:shard1 r:core_node3 x:testschemaapi_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50494.solr.core.testschemaapi.shard1.replica_n1' (registry 'solr.core.testschemaapi.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27d8e41f
   [junit4]   2> 1674184 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [[testschemaapi_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.schema.TestManagedSchemaAPI_3E461CD8626019B7-001/tempDir-001/node2/./testschemaapi_shard1_replica_n1/data/]
   [junit4]   2> 1674324 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1674324 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1674341 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1674345 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1674345 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1674351 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1674351 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1674358 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1674361 INFO  (searcherExecutor-22162-thread-1-processing-n:127.0.0.1:50494_solr) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674368 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@731a3571
   [junit4]   2> 1674368 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testschemaapi.shard1.replica_n1 tag=SolrCore@731a3571
   [junit4]   2> 1674368 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@214c45b0: rootName = solr_50494, domain = solr.core.testschemaapi.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testschemaapi.shard1.replica_n1/com.codahale.metrics.MetricRegistry@31575b12
   [junit4]   2> 1674370 INFO  (searcherExecutor-22162-thread-1-processing-n:127.0.0.1:50494_solr) [n:127.0.0.1:50494_solr     ] o.a.s.c.SolrCore [testschemaapi_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1674387 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testschemaapi.shard1.leader tag=SolrCore@731a3571
   [junit4]   2> 1674396 INFO  (qtp2146037733-32407) [n:127.0.0.1:50494_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=testschemaapi_shard1_replica_n1&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=573
   [junit4]   2> 1674402 INFO  (qtp1463187051-32395) [n:127.0.0.1:50492_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testschemaapi&action=RELOAD&wt=javabin&version=2} status=0 QTime=587
   [junit4]   2> 1674411 INFO  (qtp1463187051-32396) [n:127.0.0.1:50492_solr c:testschemaapi s:shard1 r:core_node4 x:testschemaapi_shard1_replica_n2 ] o.a.s.s.SchemaManager managed schema loaded . version : 1 
   [junit4]   

[...truncated too long message...]

 2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	19	/solr/overseer/queue
   [junit4]   2> 	12	/solr/overseer/collection-queue-work
   [junit4]   2> 	7	/solr/live_nodes
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 
   [junit4]   2> 1676981 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:50485
   [junit4]   2> 1676981 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:50485
   [junit4]   2> 1676981 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 50485
   [junit4]   2> 1676981 INFO  (SUITE-TestManagedSchemaAPI-seed#[3E461CD8626019B7]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> May 30, 2020 7:41:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> May 30, 2020 7:42:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaAPI: 
   [junit4]   2>    1) Thread[id=32363, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaAPI]
   [junit4]   2>         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]   2> May 30, 2020 7:42:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=32363, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaAPI]
   [junit4]   2> 1687141 ERROR (SessionTracker) [     ] o.a.z.s.ZooKeeperCriticalThread Severe unrecoverable error, from thread : SessionTracker
   [junit4]   2>           => java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.base/java.lang.Thread.sleep(Native Method)
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method) [?:?]
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) ~[zookeeper-3.5.7.jar:3.5.7]
   [junit4]   2> May 30, 2020 7:42:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {_root_=Lucene84, id=FST50, myNewField1=FST50, myNewField=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=957, maxMBSortInHeap=5.229719969155239, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@23a70902), locale=tg, timezone=Brazil/West
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 13.0.2 (64-bit)/cpus=6,threads=1,free=100434736,total=417689600
   [junit4]   2> NOTE: All tests run in this JVM: [SuggestComponentTest, TestRecovery, TestSort, TestHashQParserPlugin, RollingRestartTest, NestedAtomicUpdateTest, BlockDirectoryTest, TestSlowCompositeReaderWrapper, TestComponentsName, TestCloudPivotFacet, AuditLoggerIntegrationTest, TestJsonFacetRefinement, TestSerializedLuceneMatchVersion, PeerSyncWithIndexFingerprintCachingTest, TestSimUtils, TestDefaultStatsCache, IndexBasedSpellCheckerTest, HttpPartitionWithTlogReplicasTest, ShardRoutingTest, TestSearcherReuse, TestFieldCacheVsDocValues, DisMaxRequestHandlerTest, TestValueSourceCache, TestCollationFieldDocValues, TestJmxIntegration, TestLegacyField, JavabinLoaderTest, CoreAdminCreateDiscoverTest, TestCoreDiscovery, TestBinaryField, TestNonDefinedSimilarityFactory, StatsReloadRaceTest, SearchRateTriggerTest, AsyncCallRequestStatusResponseTest, TestSolrDeletionPolicy1, SpatialHeatmapFacetsTest, ScriptEngineTest, TestScoreJoinQPNoScore, CurrencyFieldTypeTest, TestHighlightDedupGrouping, TestLeaderElectionWithEmptyReplica, DocValuesTest, TestDocBasedVersionConstraints, ResponseHeaderTest, HdfsNNFailoverTest, SecurityConfHandlerTest, RAMDirectoryFactoryTest, MetricsHistoryWithAuthIntegrationTest, TestSortableTextField, TestOnReconnectListenerSupport, DeleteNodeTest, TestStressLiveNodes, SortByFunctionTest, LargeFieldTest, TriLevelCompositeIdRoutingTest, TestEmbeddedSolrServerSchemaAPI, UpdateLogTest, SolrXmlInZkTest, TestGeoJSONResponseWriter, TestMinHashQParser, SearchHandlerTest, NodeAddedTriggerIntegrationTest, RuleEngineTest, TestExpandComponent, SolrShardReporterTest, SampleTest, TestExceedMaxTermLength, TestDynamicLoadingUrl, ManagedSchemaRoundRobinCloudTest, OverseerSolrResponseTest, ZkFailoverTest, URLClassifyProcessorTest, SolrGraphiteReporterTest, SparseHLLTest, ExplicitHLLTest, TestCoreAdminApis, DistributedFacetPivotWhiteBoxTest, BadCopyFieldTest, TestLocalFSCloudBackupRestore, TestSolrCLIRunExample, SolrCoreTest, MoreLikeThisComponentTest, CSVRequestHandlerTest, TestSuggestSpellingConverter, TestClusterProperties, SchemaApiFailureTest, ClassificationUpdateProcessorTest, LukeRequestHandlerTest, TestDocumentBuilder, TestJoin, TestRandomDVFaceting, TestSimpleTrackingShardHandler, TestSolrCoreProperties, TestSolrTestCaseJ4, TestTolerantSearch, TestTrie, PathHierarchyTokenizerFactoryTest, ProtectedTermFilterFactoryTest, TestCharFilters, TestDeprecatedFilters, TestLuceneMatchVersion, TestReversedWildcardFilterFactory, TokenizerChainTest, TestLuceneIndexBackCompat, AliasIntegrationTest, CollectionPropsTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DistributedVersionInfoTest, LeaderElectionTest, LeaderFailureAfterFreshStartTest, NestedShardedAtomicUpdateTest, RecoveryZkTest, RemoteQueryErrorTest, RoutingToNodesWithPropertiesTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudRecovery, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestLocalStatsCacheCloud, TestPullReplica, TestPullReplicaErrorHandling, TestSolrCloudWithDelegationTokens, TestTolerantUpdateProcessorRandomCloud, CollectionsAPIDistributedZkTest, AutoAddReplicasPlanActionTest, IndexSizeTriggerMixedBoundsTest, ScheduledTriggerTest, TriggerEventQueueTest, TriggerIntegrationTest, StressHdfsTest, TestClusterStateMutator, ZkCollectionPropsCachingTest, PluginInfoTest, TestConfig, TestConfigOverlay, TestLazyCores, MoreLikeThisHandlerTest, RequestLoggingTest, SearchHandlerTest, TestBlobHandler, CoreAdminOperationTest, CoreAdminRequestStatusTest, MBeansHandlerTest, DistributedSuggestComponentTest, ShardsWhitelistTest, TermVectorComponentTest, TermsComponentTest, HighlighterTest, TestUnifiedSolrHighlighter, TestMultiValuedNumericRangeQuery, MetricsConfigTest, SolrJmxReporterCloudTest, SolrRrdBackendFactoryTest, RegexBytesRefFilterTest, SimpleFacetsTest, SubstringBytesRefFilterTest, TestFacetMethods, TestFaceting, TestMacros, JSONWriterTest, TestGraphMLResponseWriter, TestPushWriter, TestRawResponseWriter, TestChildDocTransformerHierarchy, TestSubQueryTransformer, TestSubQueryTransformerDistrib, EnumFieldTest, PolyFieldTest, PreAnalyzedFieldTest, RequiredFieldsTest, TestCloudManagedSchema, TestManagedSchemaAPI]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestManagedSchemaAPI -Dtests.seed=3E461CD8626019B7 -Dtests.slow=true -Dtests.locale=tg -Dtests.timezone=Brazil/West -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J4 | TestManagedSchemaAPI (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestManagedSchemaAPI: 
   [junit4]    >    1) Thread[id=32363, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestManagedSchemaAPI]
   [junit4]    >         at java.base@13.0.2/java.lang.Thread.sleep(Native Method)
   [junit4]    >         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3E461CD8626019B7]:0)
   [junit4] Completed [769/913 (2!)] on J4 in 30.39s, 1 test, 1 error <<< FAILURES!

[...truncated 52759 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6092/consoleText

[repro] Revision: a6a02ac0e5116f4cf061fd3afeb930c3cc379f5a

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC"
[repro] Repro line:  ant test  -Dtestcase=TestManagedSchemaThreadSafety -Dtests.seed=3E461CD8626019B7 -Dtests.slow=true -Dtests.locale=ga-IE -Dtests.timezone=Europe/Chisinau -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=TestManagedSchemaAPI -Dtests.seed=3E461CD8626019B7 -Dtests.slow=true -Dtests.locale=tg -Dtests.timezone=Brazil/West -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestManagedSchemaThreadSafety
[repro]       TestManagedSchemaAPI
[repro] ant compile-test

[...truncated 3147 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.TestManagedSchemaThreadSafety|*.TestManagedSchemaAPI" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=3E461CD8626019B7 -Dtests.slow=true -Dtests.locale=ga-IE -Dtests.timezone=Europe/Chisinau -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 106 lines...]
[repro] Failures w/original seeds:
[repro]   0/5 failed: org.apache.solr.schema.TestManagedSchemaAPI
[repro]   0/5 failed: org.apache.solr.schema.TestManagedSchemaThreadSafety
[repro] Exiting with code 0

[...truncated 73 lines...]